Polars

oleander’s Polars integration lets you run Polars workloads directly against your Iceberg lake tables - no infrastructure to provision. Workloads execute in an isolated sandbox, or across multiple workers in distributed mode. A DataFrame script always selects Polars: the query router sends anything passed as script here, whatever engine you asked for. SQL only reaches Polars when you request it explicitly.

Modes

SQL query

Run a Polars SQL expression against one or more registered lake tables. This is the fastest way to explore data without writing Python.

Python script

Write a Python script using the Polars DataFrame API. oleander injects the runtime, auth, and catalog - your script only needs to assign result. The following are available in scope without any imports:

Variable	Type	Description
`pl`	module	The `polars` module
`scan(table)`	function	Returns a `LazyFrame` for `namespace.table` with credentials wired in
`params`	`dict`	Key-value parameters passed at runtime
`catalog`	object	The underlying pyiceberg catalog (advanced use)

Organization environment variables are available through os.environ. Your script must assign its output to result - a Polars LazyFrame or DataFrame. oleander handles .collect() and distributed dispatch; do not call them in your script. If your script builds a table name at runtime, such as scan(params["table"]), also declare it in tables so the query router can size the sandbox.

table = params.get("table", "default.flowers")
limit = int(params.get("limit", 50))

flowers = scan(table)

result = (
    flowers.group_by("species")
    .agg(
        pl.len().alias("count"),
        pl.col("sepal_length").mean().round(2).alias("avg_sepal_length"),
        pl.col("petal_length").mean().round(2).alias("avg_petal_length"),
    )
    .sort("avg_sepal_length", descending=True)
    .head(limit)
)

Runtime

Sandboxes run Polars 2.1. In Polars SQL, a decimal literal such as 1.1 is a Decimal rather than a Float64. When a run writes to an existing table, columns are cast to the table’s types. Polars SQL does not support INSERT, UPDATE, or some window frames. On engine: auto, the query router sends those queries to another engine.

Distributed execution

Distributed mode runs your workload across multiple workers instead of a single sandbox, for datasets that exceed sandbox memory. A distributed run always writes to a destination table and requires the Pro++ plan.

Setting	Values
Instance type	`t4g.medium` (default), `t4g.large`, `m6g.large`, `m6g.xlarge`, `r6g.xlarge`
Cluster size	1 to 10 workers

Distributed runs are billed as distributed compute from cluster startup until the run finishes. See pricing.

Saving results

Results can be written back to a table in your Iceberg catalog. Choose overwrite to replace the table or append to add rows to an existing one.

Lineage

Every Polars run, SQL or script, captures lineage, including column lineage, through openlineage-polars. Inputs are the tables the run read and the output is the table it wrote, in the same graph as the other engines.

CLI & API

Polars workloads can also be triggered programmatically:

CLI: oleander polars
API: POST /api/v1/warehouse/polars

Getting Started

Platform

IAM

Connections

Integrations

Coding with agents

CLI

External Compute

Modes

SQL query

Python script

Runtime

Distributed execution

Saving results

Lineage

CLI & API

​Modes

​SQL query

​Python script

​Runtime

​Distributed execution

​Saving results

​Lineage

​CLI & API

Modes

SQL query

Python script

Runtime

Distributed execution

Saving results

Lineage

CLI & API