# DuckDB (legacy) Source: https://docs.oleander.dev/api-reference/duckdb-legacy /api-reference/openapi.json post /api/v1/warehouse/query Legacy endpoint. Executes SQL against the oleander lake, always on DuckDB, sized by the organization's default sandbox setting rather than by the query. Prefer POST /api/v1/query, which routes to the right engine and machine; use this only for autoSaveByHash. All queries capture lineage metadata automatically. This endpoint is only available on the Pro plan. # API Reference Source: https://docs.oleander.dev/api-reference/introduction REST API for ingesting lineage events and querying the oleander lake. The oleander API covers two surfaces: lineage ingestion via the OpenLineage spec, and lake queries through the [query router](/platform/query-routing/overview), which picks between DuckDB, Bloom, Polars, and Spark. All endpoints authenticate with a Bearer token. ## Base URL ``` https://oleander.dev ``` ## Authentication ```bash theme={null} curl -H "Authorization: Bearer YOUR_API_KEY" https://oleander.dev/api/v1/events ``` Generate an API key from your [oleander settings](https://oleander.dev/app/settings/api-keys). ## Endpoints **Lineage** | Method | Endpoint | Description | | - | - | - | | `POST` | `/api/v1/lineage` | Submit an OpenLineage event | | `GET` | `/api/v1/events` | Retrieve lineage events with filtering and pagination | **Queries** | Method | Endpoint | Description | | - | - | - | | `POST` | `/api/v1/query` | Run or submit a query. oleander picks the engine and machine size. Start here. | | `POST` | `/api/v1/warehouse/query` | Legacy. Executes SQL on DuckDB, sized by the org default rather than by the query. | | `POST` | `/api/v2/spark/query` | Run a SQL query on managed Spark, writing results to an Iceberg table | | `POST` | `/api/v1/warehouse/polars` | Run Polars SQL or a Python script against your Iceberg lake | `/api/v1/query` is the endpoint the SDKs and MCP tools use. It parses the SQL, estimates the input, and routes to DuckDB, Polars, [Bloom](/platform/compute/bloom), or Spark, returning the decision in `engine_decision`. See [Query routing](/platform/query-routing/overview). The engine-specific endpoints below it remain available when you need to pin one. ## OpenLineage compatibility oleander implements the [OpenLineage](https://openlineage.io/) specification. Any OpenLineage-compatible producer - Airflow, dbt, Spark, Flink - can send events directly to `/api/v1/lineage` without modification. # Polars Source: https://docs.oleander.dev/api-reference/polars /api-reference/openapi.json post /api/v1/warehouse/polars Execute a Polars workload against your Iceberg lake tables. Supports two modes: **query** (Polars SQL run against registered tables) and **script** (user-authored Python that assigns `result` to a LazyFrame or DataFrame). Workloads run in an isolated sandbox; pass `distributed: true` to offload execution to Polars Cloud. **Beta:** Observability (lineage, traces, logs) is not yet captured for Polars workloads. **Cost note:** Distributed execution incurs Polars Cloud compute charges in addition to oleander usage. Start small and scale up as needed. # Query (routed) Source: https://docs.oleander.dev/api-reference/query-routed /api-reference/openapi.json post /api/v1/query The unified query endpoint. oleander parses the SQL, estimates how much data the referenced tables hold, and picks the engine (DuckDB, Polars, Bloom, or Spark) and machine size to match. Leave `engine` on `auto` unless you need a specific one. Interactive reads return rows on the call. A query with a `destination`, a statement that names its own target, or a read too large for an interactive engine is submitted asynchronously and returns `state: "SUBMITTED"` with a `run_id` to poll. This endpoint supersedes `/api/v1/warehouse/query`, which pins DuckDB and is sized by the org default rather than by the query. # Record event Source: https://docs.oleander.dev/api-reference/record-event /api-reference/openapi.json post /api/v1/lineage Submit an OpenLineage compliant event to record lineage metadata. This is the primary endpoint for ingesting lineage data from your pipelines and integrations. # Retrieve events Source: https://docs.oleander.dev/api-reference/retrieve-events /api-reference/openapi.json get /api/v1/events List lineage events. Supports filtering by time range, run, state, namespace, job, and integrations. Results can be paginated and sorted. # Spark Source: https://docs.oleander.dev/api-reference/spark /api-reference/openapi.json post /api/v2/spark/query Execute a SQL query against the oleander lake using managed Spark. The query runs as a named job in the specified namespace. Results are written to `output_table` as an Iceberg table. Lineage is captured automatically for every run. # Architecture Source: https://docs.oleander.dev/architecture Learn how your data stack connects to oleander through OpenLineage and OpenTelemetry, and how telemetry builds the context graph. oleander is built in three layers: a **telemetry ingestion layer** that collects OpenLineage and OpenTelemetry events from your stack, a **managed lake** that stores telemetry and user data in open formats, and a **context graph** that unifies runs, datasets, costs, and lineage into a single versioned, queryable object model with built-in **time travel** and a knowledge layer that learns from your data team. **Serverless Spark** runs on oleander's managed infrastructure, isolated, fully provisioned, no clusters to configure. Write to the Iceberg catalog from a Spark job or trigger a job on a dataset refresh, a partition write, or an upstream change via a webhook. Every read and write captures lineage automatically in real time, so agents always have the full picture when investigating failures, optimizing pipelines, or writing code. oleander architecture oleander architecture *** ## Telemetry oleander collects telemetry via **OpenLineage** and **OpenTelemetry**, the two open standards already running in your stack. **No agents** are deployed in your environment. **No direct access** to your infrastructure is required. Point your existing data tools at oleander's ingest endpoint to start collecting telemetry. Every event, log, and trace is written to `oleander.telemetry`. | Standard | What it captures | | - | - | | `OpenLineage` | Job runs, dataset inputs and outputs, lineage edges, schema versions | | `OpenTelemetry` | Spans, traces, logs, and resource attributes from pipeline execution | *** ## Catalog Every oleander organization is provisioned a fully managed **Iceberg** catalog named `oleander.default`. It exposes a standard Iceberg REST endpoint, making it natively compatible with Spark, DuckDB, and any Iceberg-aware tool. The catalog contains two namespaces out of the box, or you can register your own:
Namespace Purpose
oleander.telemetry Telemetry data written automatically: run\_events, traces, logs
oleander.default Your data, queryable and writable as Iceberg
\ Register an external catalog (**AWS S3 Tables** or **Snowflake Horizon**) and query it alongside your data. For more details, see Catalogs.
For example, register a **Snowflake Horizon** catalog and join pipeline failures against your `customers` table to understand which customers are impacted during a critical outage to offer credits.. ```sql theme={null} SELECT r.job_name, r.event_time, r.error_message, c.customer_name, c.tier, c.sla_threshold_minutes FROM oleander.telemetry.run_events AS r JOIN snowflake.my_database.customers AS c ON r.job_name = c.pipeline_name WHERE r.event_type = 'FAIL' AND r.event_time > NOW() - INTERVAL '24 hours' AND c.tier = 'enterprise' ORDER BY r.event_time DESC; ``` *** ## Graph oleander reads from the catalog and builds a versioned **context graph**: a unified object model where every run, dataset, schema, query, cost, and lineage edge is a first-class, interconnected node. The graph is updated automatically after every execution. You can query it as it exists now or as it existed at any point in time, diff any two runs, and trace exactly what changed and when. | Object | What it contains | | - | - | | **Runs** | Every DAG run, Spark job, dbt model: `duration`, `status`, `logs`, `lineage` | | **Datasets** | Tables and schemas versioned at every write, with upstream and downstream lineage | | **Costs** | Every warehouse query attributed to the workflow and dataset | | **Changes** | Schema diffs and PR-level impact before they cascade downstream | ### Knowledge Every node in the context graph is versioned at the time it is written. The graph maintains full history: query it as it exists now or as it existed at any point in time, diff any two runs, and trace exactly what changed and when. The graph learns from two sources: **telemetry** captured automatically from every run, cost, and schema change, and **annotations** your team adds during investigations to record context that never appears in a log. Any node in the graph can be annotated. Annotations are attached to the node as **memories** and surfaced automatically the next time oleander, an agent, or an engineer investigates that pipeline, dataset, or cost. *** ## Compute oleander provides a managed compute environment that reads from and writes to the Iceberg catalog. Lineage and cost are attributed automatically on every read and write, with no configuration required. ### Query routing Queries do not name an engine. The **[query routing layer](/platform/query-routing/overview)** parses the SQL, resolves the tables it touches, estimates the input from Iceberg snapshot metadata, and picks both the engine and the machine size. | Engine | Where work lands | | - | - | | **[Bloom](/platform/compute/bloom)** | The default for reads. DataFusion over Iceberg, local on a sandbox or distributed across sized workers. | | **[DuckDB](/platform/compute/duckdb)** | Mutations, catalogs other than `oleander`, and [BigQuery, Snowflake, and Postgres](/connections/bigquery) connections - the only engine that attaches them. | | **[Polars](/platform/compute/polars)** | DataFrame scripts and in-memory transformation. | | **[Spark](/platform/compute/spark)** | Iceberg DDL, writes to a destination, and inputs too large for a sandbox. | Every decision returns a reason trail alongside the result, and `explain` returns the decision on its own without spending compute. ### Serverless Spark Submit PySpark applications to fully managed infrastructure. No clusters to provision, no configuration required. Spark jobs read from and write to both `oleander.default` and external Iceberg catalogs. Lineage and cost flow into the context graph from the first run. ```bash theme={null} oleander spark submit my_job.py ``` > Bring your own Spark cluster. Connect **Databricks**, **EMR**, or **Dataproc** to oleander's **Iceberg** catalog endpoint and telemetry, Spark execution plans, and lineage are captured in the context graph automatically. *** ## Query Engineers and agents query the same context graph and catalog from wherever they work. | Interface | What it does | | - | - | | **MCP server** | Coding agents (Cursor, Copilot, Claude, BYO) query the context graph directly | | **API** | Programmatic access to the graph, catalog, and SQL proxy | | **CLI** | Terminal-based queries, Spark submission, and lake management | | **SDK** | Custom integrations and automated workflows | | **SQL** | Query the telemetry lake and user tables directly | > The SQL proxy lets you query [BigQuery](/connections/bigquery), [Snowflake](/connections/snowflake), and [Postgres](/connections/postgres) directly. Queries are attributed to the workflow that triggered them and appear in the context graph. # OpenLineage producers Source: https://docs.oleander.dev/build-deploy Connect any OpenLineage-compatible tool to oleander for automatic lineage tracking. oleander implements the [OpenLineage](https://openlineage.io/) specification, which means it works out of the box with any tool that emits OpenLineage events - no custom plugins or proprietary agents required. Point any OpenLineage producer at your oleander endpoint with your API key, and lineage, traces, and metadata will start flowing automatically. ```bash theme={null} # The oleander OpenLineage endpoint https://oleander.dev/api/v1/lineage # Authenticate with your API key Authorization: Bearer ``` ## Supported integrations Guides are available for the most common OpenLineage producers: Track lineage across your Airflow DAGs using the built-in OpenLineage provider. Capture dataset-level lineage from your Spark jobs via the OpenLineage Spark listener. Emit lineage from every dbt model run using the `dbt-ol` wrapper. ## Any OpenLineage producer If your tool isn't listed above, check the [OpenLineage integrations directory](https://openlineage.io/docs/integrations/about) - the full list includes Flink, Dagster, Prefect, Great Expectations, and more. Any integration that supports a configurable transport URL will work with oleander. You can also send events directly via the [API](/api-reference/introduction) or the [CLI](/cli/introduction). # Env Source: https://docs.oleander.dev/cli/env Manage organization-level environment variables from the CLI. Environment variables set here are available to Spark jobs running on the oleander-managed cluster. They are stored encrypted and injected at runtime. ## List environment variables ```bash theme={null} oleander env list ``` Pass `--json` for machine-readable output: ```bash theme={null} oleander env list --json ``` Example output: ``` Environment variables - name: DATABASE_URL id: ev_abc123 updated: 2026-05-01T12:00:00Z - name: API_SECRET id: ev_def456 updated: 2026-05-10T09:30:00Z ``` ## Set an environment variable ```bash theme={null} oleander env set ``` Names are uppercased automatically. Creating a variable with an existing name replaces it. **Example:** ```bash theme={null} oleander env set DATABASE_URL "postgresql://host:5432/mydb" ``` ## Delete an environment variable ```bash theme={null} oleander env delete ``` **Example:** ```bash theme={null} oleander env delete DATABASE_URL ``` # Intro Source: https://docs.oleander.dev/cli/introduction Install and configure the oleander CLI to manage Spark jobs, query the lake, manage catalogs and tables, handle environment variables, and save queries from your terminal. The oleander CLI lets you manage Spark artifacts and jobs, run SQL queries against the lake, browse catalogs, register infrastructure, and launch a DuckDB terminal from the command line. Queries go through the [query router](/platform/query-routing/overview), so the CLI picks the engine and machine size for you. ## Installation ```bash Homebrew theme={null} brew tap OleanderHQ/tap brew install oleander-cli ``` ```bash Ubuntu (APT) theme={null} sudo apt-get update sudo apt-get install -y curl ca-certificates gnupg curl -fsSL https://oleander-cli-releases.s3.amazonaws.com/keys/oleander-archive-keyring.gpg \ | sudo tee /usr/share/keyrings/oleander-archive-keyring.gpg >/dev/null sudo tee /etc/apt/sources.list.d/oleander.sources >/dev/null <<'EOF' Types: deb URIs: https://oleander-cli-releases.s3.amazonaws.com/apt Suites: stable Components: main Signed-By: /usr/share/keyrings/oleander-archive-keyring.gpg EOF sudo apt-get update sudo apt-get install -y oleander-cli ``` ## Configuration Authenticate with an API key from your [oleander settings](https://oleander.dev/app/settings/api-keys). Pass a profile name. In a terminal, the CLI prompts for the key and stores it in `~/.oleander/config.json`. The first profile you configure becomes the current profile. ```bash theme={null} oleander configure --profile production ``` Outside a terminal, set `OLEANDER_API_KEY` or pipe the key on stdin: ```bash theme={null} OLEANDER_API_KEY=... oleander configure --profile ci ``` You can also run `oleander configure` without flags to enter your key interactively. The CLI stores the API key in `~/.oleander/config.json`. ## Upgrade Upgrade to the latest release: ```bash theme={null} oleander upgrade ``` Check whether an update is available without installing it: ```bash theme={null} oleander upgrade --check ``` ## Commands ### General | Command | Description | | - | - | | `oleander configure` | Set your API key | | `oleander upgrade` | Upgrade to the latest release | ### Lake | Command | Description | | - | - | | `oleander query` | Run a read against the lake; oleander picks the engine and machine size | | `oleander query submit` | Run a query that writes - a `SELECT` plus `--destination`, or a statement that names its own target | | `oleander polars` | Run a Polars SQL query or Python script against the lake | | `oleander duckdb` | Launch a DuckDB terminal connected to your lake | ### Catalogs | Command | Description | | - | - | | `oleander catalogs list` | List registered catalogs | | `oleander catalogs register` | Register an S3 Tables catalog | | `oleander catalogs credentials` | Get short-lived credentials for a catalog | | `oleander catalogs namespaces list` | List namespaces in a catalog | | `oleander catalogs namespaces create` | Create a namespace | | `oleander catalogs tables list` | List tables in a namespace | | `oleander catalogs tables describe` | Describe a table schema | | `oleander catalogs tables create` | Create a new Iceberg table | | `oleander catalogs tables delete` | Delete a table | | `oleander catalogs tables load` | Load data from a file or URI into a new table | | `oleander catalogs tables schema add` | Add columns to a table | | `oleander catalogs tables schema rename` | Rename a column | | `oleander catalogs tables schema drop` | Drop columns from a table | ### Spark | Command | Description | | - | - | | `oleander spark init` | Initialize a PySpark job workspace | | `oleander spark jobs list` | List uploaded Spark artifacts | | `oleander spark jobs upload` | Upload a Spark artifact | | `oleander spark jobs get` | Download a Spark artifact's source | | `oleander spark jobs delete` | Delete a Spark artifact | | `oleander spark jobs submit` | Submit a Spark job | | `oleander spark jobs stop` | Stop a running Spark job | | `oleander spark clusters list` | List registered Spark clusters | | `oleander spark clusters register` | Register an external Spark cluster | ### Env | Command | Description | | - | - | | `oleander env list` | List organization environment variables | | `oleander env set` | Set an environment variable | | `oleander env delete` | Delete an environment variable | ### Queries | Command | Description | | - | - | | `oleander queries list` | List saved queries | | `oleander queries get` | Get a saved query by name | | `oleander queries save` | Save a named query | See [Spark](/cli/spark), [Lake](/cli/lake), [Polars](/cli/polars), [Env](/cli/env), and [Queries](/cli/queries) for detailed usage. # Lake Source: https://docs.oleander.dev/cli/lake Query the lake, explore catalogs, register catalogs, and launch a DuckDB terminal from the CLI. Run SQL queries, browse your catalogs and tables, register S3 Tables catalogs, and open a DuckDB terminal connected to your lake from the command line. For a full overview of the lake, see the [DuckDB](/platform/compute/duckdb) documentation. ## Query `oleander query` sends SQL through the [query router](/platform/query-routing/overview): oleander parses it, estimates how much data the referenced tables hold, and picks the engine and machine size to match. There is nothing to size by hand. ```bash theme={null} oleander query "" ``` **Example:** ```bash theme={null} oleander query "SELECT * FROM oleander.default.flowers LIMIT 10" ``` Results are printed as a formatted table with row counts and execution time, under a block naming the engine that ran the query, the sandbox or cluster shape, the estimated input, and the router's reasons. `oleander query` is read-only. Anything that changes data - a `SELECT` you want written to a table, or an `INSERT`, `UPDATE`, `DELETE`, `MERGE`, or DDL statement - goes to [`oleander query submit`](#write-results-with-query-submit). Mutating SQL is refused before the request leaves your machine. ### Explain before you run `--explain` prints the engine, estimated input size, and reason trail without executing anything or spending compute. Worth doing before a query you expect to be large. ```bash theme={null} oleander query "SELECT * FROM oleander.default.events" --explain ``` ``` ✔ Planned (nothing ran, no compute spent) Engine: bloom (interactive) · 4 vCPU / 8 GB · ~3.4 GiB input Tables: oleander.default.events • Estimated input 3.4 GiB from 1/1 Iceberg table snapshot(s). • Medium input: scaling the Bloom sandbox instead of switching engines. • Sandbox sized to 4 vCPU / 8 GB. ``` The same block prints under every result, so the engine choice is visible where the query ran. ### Pick an engine yourself Leave the engine on `auto` unless you have a reason. `--engine` accepts `auto`, `duckdb`, `polars`, or `bloom`; Spark is not an option here because it never returns rows. ```bash theme={null} oleander query "SELECT * FROM oleander.default.flowers LIMIT 10" --engine duckdb ``` An engine that cannot serve the query returns a capability error rather than rerouting. ### Polars scripts Pass `--script` instead of SQL to run a Python [Polars](/platform/compute/polars) script that assigns `result`, declaring the tables it reads with `--table`. This forces the Polars engine. ```bash theme={null} oleander query --script ./job.py --table events=oleander.default.events ``` ### Write results with `query submit` `oleander query submit` takes everything that changes data: a `SELECT` plus `--destination`, or a statement that names its own target. ```bash theme={null} # Write a result set to a table oleander query submit "SELECT * FROM oleander.default.flowers" \ --destination default.flower_sample # Append instead of replacing, and block until the run finishes oleander query submit "SELECT * FROM oleander.default.flowers" \ --destination default.flower_sample \ --write-mode append \ --wait # A statement that writes on its own needs no destination oleander query submit "DELETE FROM oleander.default.flower_sample WHERE species IS NULL" ``` `--write-mode` defaults to `overwrite`, which replaces the destination table's contents. Whether a write finishes on the call is the router's choice, not yours, so read the state it reports: | Output | Meaning | | - | - | | `Write complete` | The write already landed. Nothing to poll. | | A run id | A job is running and the write is not visible yet. | Add `--wait` to block on a submitted run until it reaches a terminal state; `--poll-interval` (default 10s) and `--timeout` (default 600s) control the polling. A run that fails or is aborted exits non-zero. ### Options | Option | Applies to | Description | | - | - | - | | `--engine ` | both | `auto` (default), `duckdb`, `polars`, `bloom`; `spark` on `submit` only | | `--explain` | both | Print the routing decision without running or submitting anything | | `--json` | both | Print the raw API response as JSON | | `--script ` | both | Polars script that assigns `result`, run instead of SQL | | `--table ` | both | Table a `--script` reads. Repeatable. | | `--destination ` | `submit` | Table to write the result to | | `--write-mode ` | `submit` | `overwrite` (default) or `append` | | `--wait` | `submit` | Block until an asynchronous run reaches a terminal state | | `--poll-interval ` | `submit` | Seconds between `--wait` polls (default `10`) | | `--timeout ` | `submit` | Seconds to wait before giving up (default `600`) | A 402 or 403 on a query is a billing decision, not a transient failure. The CLI prints what it asks for - settle billing, upgrade the plan, add a card, or run a read on `--engine duckdb`. Retrying the same query fails identically. `oleander query --save` and `--table-name` are gone: a read no longer writes a table. Use `oleander query submit --destination` instead. ## Catalogs ### List catalogs List all available catalogs in your organization: ```bash theme={null} oleander catalogs list ``` ### Register an S3 Tables catalog Register a catalog backed by S3 Tables: ```bash theme={null} oleander catalogs register \ --region \ --account-id \ --bucket \ --role-arn ``` `--type` is optional and defaults to `s3tables`. ### Get catalog credentials Retrieve short-lived credentials for a registered catalog: ```bash theme={null} oleander catalogs credentials ``` Pass `--json` to get raw JSON output (useful for scripting): ```bash theme={null} oleander catalogs credentials my-catalog --json ``` ### List namespaces ```bash theme={null} oleander catalogs namespaces list --catalog ``` Omit `--catalog` to use the `oleander` default catalog. Pass `--json` for JSON output. ### Create a namespace ```bash theme={null} oleander catalogs namespaces create --catalog ``` `--catalog` defaults to `oleander`. ### List tables List tables in a catalog namespace: ```bash theme={null} oleander catalogs tables list --catalog --namespace ``` If you omit `--catalog` or `--namespace`, the CLI prompts you interactively. ### Describe a table View the schema of a table: ```bash theme={null} oleander catalogs tables describe \ --catalog \ --namespace \ --table ``` If you omit any of `--catalog`, `--namespace`, or `--table`, the CLI prompts you interactively. ### Create a table Create a new Iceberg table from a JSON schema definition: ```bash theme={null} oleander catalogs tables create \ --catalog \ --namespace \ --schema '{"type":"struct","fields":[{"id":1,"name":"id","required":true,"type":"long"},{"id":2,"name":"name","required":false,"type":"string"}]}' ``` Pass a file path prefixed with `@` or just a path to load the schema from a file: ```bash theme={null} oleander catalogs tables create events \ --catalog oleander \ --namespace default \ --schema ./schema.json ``` Optionally include a partition spec: ```bash theme={null} oleander catalogs tables create events \ --schema ./schema.json \ --partition-spec ./partition.json ``` `--catalog` defaults to `oleander`. `--namespace` defaults to `default`. ### Delete a table ```bash theme={null} oleander catalogs tables delete \ --catalog \ --namespace ``` `--catalog` defaults to `oleander`. `--namespace` defaults to `default`. ### Load data into a table Create a new table and populate it from a local file or a remote URI. The source can be a local Parquet, CSV, or JSON file, or an S3/HTTPS URI. ```bash theme={null} oleander catalogs tables load \ --table \ --source \ --file-type parquet ``` **Local file example:** ```bash theme={null} oleander catalogs tables load \ --table flowers \ --source ./data/flowers.parquet \ --file-type parquet ``` **Remote URI example:** ```bash theme={null} oleander catalogs tables load \ --table flowers \ --source s3://my-bucket/data/flowers.parquet \ --file-type parquet ``` **CSV example:** ```bash theme={null} oleander catalogs tables load \ --table orders \ --source ./data/orders.csv \ --file-type csv ``` | Flag | Description | | - | - | | `--table ` | Destination table name (required) | | `--source ` | Local file path or remote URI (required) | | `--file-type ` | `parquet`, `csv`, or `json` (required) | | `--catalog ` | Catalog name (defaults to `oleander`) | | `--namespace ` | Namespace (defaults to `default`) | **How each file type is read:** * **CSV**: the first row is used as the column names, and column types are inferred from the values (integers, decimals, timestamps, booleans, and so on) rather than loaded as strings. A quote inside a quoted field is written as two double quotes, per RFC 4180. A CSV without a header row is not supported: its first data row would become the column names. * **Parquet**: column names and types come from the file's schema. * **JSON**: newline-delimited JSON (one object per line). Column names come from the object keys and types are inferred. The load runs as a Spark job. The table is queryable once the run reaches `COMPLETE`; the run appears on the [Pipelines](/platform/observability/overview) page like any other job. ### Schema evolution Add, rename, or drop columns on an existing table. Pass the operation as a JSON body - either inline or from a file. **Add columns:** ```bash theme={null} oleander catalogs tables schema add \ --catalog oleander \ --namespace default \ --table events \ --input '{"columns":[{"id":3,"name":"created_at","required":false,"type":"timestamptz"}]}' ``` **Rename a column:** ```bash theme={null} oleander catalogs tables schema rename \ --catalog oleander \ --namespace default \ --table events \ --input '{"from":"created_at","to":"event_time"}' ``` **Drop columns:** ```bash theme={null} oleander catalogs tables schema drop \ --catalog oleander \ --namespace default \ --table events \ --input '{"columns":["event_time"]}' ``` All three schema commands accept `--input` as either an inline JSON string or a path to a JSON file. ## DuckDB terminal Launch a DuckDB terminal pre-configured with your registered catalogs: ```bash theme={null} oleander duckdb ``` The CLI loads registered `lakekeeper` and `s3tables` catalogs into DuckDB before opening the interactive shell. ```sql theme={null} SHOW ALL TABLES; ``` The `duckdb` command requires [DuckDB](https://duckdb.org/docs/installation) to be installed on your machine. # Polars Source: https://docs.oleander.dev/cli/polars Run Polars workloads against your lake - SQL queries or Python scripts - from the CLI. Polars is in **beta**. Observability (lineage, traces, logs) is not yet available for Polars workloads. The `oleander polars` command runs Polars workloads against your Iceberg lake tables. It supports two modes: * **Query mode** - execute a Polars SQL expression across one or more tables * **Script mode** - run a Python script that uses the Polars API directly Workloads run in an isolated Daytona sandbox. Pass `--distributed` to offload execution to Polars Cloud. ## Query mode Run a Polars SQL query against one or more lake tables. Register each table with `--table`. ```bash theme={null} oleander polars "SELECT * FROM flowers LIMIT 25" \ --table flowers=default.flowers ``` Use `alias=namespace.table` to register a table under a short alias. Multiple `--table` flags are supported: ```bash theme={null} oleander polars "SELECT f.species, m.mean_sepal_length FROM flowers f JOIN measurements m USING (species) LIMIT 50" \ --table flowers=default.flowers \ --table measurements=default.flower_measurements ``` ## Script mode Pass a Python file with `--script`. oleander injects the Polars runtime, auth, and catalog - your script only needs to assign a result. ```bash theme={null} oleander polars --script ./my_analysis.py \ --param table=default.flowers \ --param limit=100 ``` ### Python template The following variables are available in your script without any imports: | Variable | Description | | - | - | | `pl` | The `polars` module | | `scan(table)` | Returns a `LazyFrame` for the given `namespace.table` with credentials already wired in | | `params` | `dict` of values passed via `--param key=value` | | `catalog` | The underlying pyiceberg catalog (advanced use) | Your script must assign its output to `result` - a Polars `LazyFrame` or `DataFrame`. Do not call `.collect()` or `.remote()` yourself; oleander handles execution. ```python theme={null} # my_analysis.py table = params.get("table", "default.flowers") limit = int(params.get("limit", 50)) flowers = scan(table) result = ( flowers.group_by("species") .agg( pl.len().alias("count"), pl.col("sepal_length").mean().round(2).alias("avg_sepal_length"), pl.col("petal_length").mean().round(2).alias("avg_petal_length"), pl.col("sepal_width").mean().round(2).alias("avg_sepal_width"), ) .sort("avg_sepal_length", descending=True) .head(limit) ) ``` Run it: ```bash theme={null} oleander polars --script ./my_analysis.py \ --param table=default.flowers \ --param limit=25 ``` ## Saving results Use `--save` to write the result to a table in your lake. The destination is a `namespace.table` identifier. ```bash theme={null} oleander polars --script ./my_analysis.py --save default.flower_summary ``` Append to an existing table instead of replacing it: ```bash theme={null} oleander polars --script ./my_analysis.py \ --save default.flower_summary \ --save-mode append ``` ## Distributed execution Pass `--distributed` to run the workload on Polars Cloud instead of in the local sandbox. Use `--instance-type` and `--cluster-size` to control the compute. ```bash theme={null} oleander polars --script ./my_analysis.py \ --distributed \ --instance-type t4g.large \ --cluster-size 4 ``` Distributed execution incurs Polars Cloud compute costs in addition to your oleander usage. Large `--cluster-size` values or long-running scripts can accumulate significant charges. Start with the default instance type and a small cluster size, then scale up as needed. ## Options | Option | Description | | - | - | | `--script ` | Path to a Python script (activates script mode) | | `--table ` | Register a table for query mode. Repeatable. | | `--param ` | Pass a parameter to the script via `params`. Repeatable. | | `--distributed` | Run on Polars Cloud instead of a local sandbox | | `--instance-type ` | Polars Cloud instance type (default: `t4g.medium`) | | `--cluster-size ` | Number of nodes for distributed execution | | `--vcpus ` | Sandbox vCPU tier for local execution: `2`, `4`, `8`, `16`, or `32` | | `--save ` | Write results to a lake table | | `--save-mode ` | `overwrite` (default) or `append` | | `--catalog ` | Catalog name (default: `oleander`) | # Queries Source: https://docs.oleander.dev/cli/queries Save, retrieve, and manage named SQL queries from the CLI. Saved queries let you store and share frequently used SQL expressions. They are scoped to your organization. ## List saved queries ```bash theme={null} oleander queries list ``` Pass `--json` for machine-readable output: ```bash theme={null} oleander queries list --json ``` Example output: ``` Saved queries - name: top_customers id: sq_abc123 updated: 2026-05-01T12:00:00Z by: alice query: SELECT customer_id, sum(revenue) AS total FROM orders ... ``` ## Get a saved query Print the full details of a saved query by name: ```bash theme={null} oleander queries get ``` To print only the SQL (useful for piping into `oleander query`): ```bash theme={null} oleander queries get --query-only ``` **Example - run a saved query directly:** ```bash theme={null} oleander query "$(oleander queries get top_customers --query-only)" ``` To write its results to a table instead of printing them, send the same SQL to `oleander query submit`: ```bash theme={null} oleander query submit "$(oleander queries get top_customers --query-only)" \ --destination default.top_customers_snapshot ``` `oleander queries save` stores the **SQL text** under a name. [`oleander query submit --destination`](/cli/lake#write-results-with-query-submit) writes the **result set** to a table. They are different things. Pass `--json` to get the raw JSON response: ```bash theme={null} oleander queries get top_customers --json ``` ## Save a query ```bash theme={null} oleander queries save "" ``` If a query with the same name exists it is replaced. **Example:** ```bash theme={null} oleander queries save top_customers \ "SELECT customer_id, sum(revenue) AS total FROM oleander.default.orders GROUP BY 1 ORDER BY 2 DESC LIMIT 20" ``` # Spark Source: https://docs.oleander.dev/cli/spark Initialize workspaces, upload artifacts, submit jobs, and register clusters from the CLI. Run your Spark applications on oleander-managed infrastructure or on your own registered clusters. For a full overview of Spark on oleander, see the [Spark documentation](/platform/compute/spark). ## Oleander-managed Spark ### Initialize a PySpark workspace Create a new PySpark job workspace: ```bash theme={null} oleander spark init ``` **Example:** ```bash theme={null} oleander spark init my-job ``` You can also initialize the current directory: ```bash theme={null} oleander spark init . ``` Pass flags to skip the interactive prompts: ```bash theme={null} oleander spark init my-job \ --entrypoint-name pipeline.py \ --custom-package-name mylib ``` | Flag | Description | | - | - | | `--entrypoint-name ` | Name for the main entrypoint file (must end with `.py`) | | `--custom-package-name ` | Name for the custom Python package directory | The initialized workspace includes: * `entrypoint.py` as the Spark job entrypoint * `mylib/` for Python modules packaged as `pyFiles` * `pyproject.toml` and `uv.lock` for project and dependency management with `uv` * `Makefile` targets for building deployable artifacts Use `uv` to manage dependencies: ```bash theme={null} uv sync --dev uv add uv add --dev ``` Use `make` to build the deployment artifacts: ```bash theme={null} make ``` This builds: * `out/pyfiles.zip` * `out/environment.tar.gz` You can also build individual artifacts: ```bash theme={null} make pyfiles make environment make rebuild ``` After building, upload and submit from the initialized workspace: ```bash theme={null} oleander spark jobs upload entrypoint.py \ --py-files out/pyfiles.zip \ --virtualenv out/environment.tar.gz ``` ```bash theme={null} oleander spark jobs submit entrypoint.py \ --namespace \ --name \ --wait ``` ### List your Spark artifacts ```bash theme={null} oleander spark jobs list ``` ### Download a Spark artifact Print the source of an uploaded artifact to stdout: ```bash theme={null} oleander spark jobs get ``` To fetch a specific version: ```bash theme={null} oleander spark jobs get --artifact-version 3 ``` | Flag | Description | | - | - | | `--artifact-version ` | Version number to retrieve (defaults to latest) | ### Upload a Spark artifact Upload a local `.py` or `.jar` artifact to oleander: ```bash theme={null} oleander spark jobs upload ``` **Example:** ```bash theme={null} oleander spark jobs upload ./transformations/process_sales_data.py ``` **JAR example:** ```bash theme={null} oleander spark jobs upload ./jobs/process_sales_data.jar ``` Every upload creates a new artifact version on the backend. Upload options: | Flag | Description | | - | - | | `--namespace` | Namespace used when uploading. Defaults to `default`. | | `--py-files` | Local `.zip` or `.egg` dependency archive uploaded alongside a Python artifact | | `--virtualenv` | Local virtual environment archive uploaded alongside a Python artifact | | `--dry-run` | Show what would be uploaded without uploading anything | ### Delete a Spark artifact ```bash theme={null} oleander spark jobs delete ``` Use the exact uploaded artifact name, including the file extension. **Example:** ```bash theme={null} oleander spark jobs delete process_sales_data.py ``` ### Submit a job Submit an uploaded artifact to the oleander-managed cluster. Use the uploaded artifact name as the `entrypoint`. ```bash theme={null} oleander spark jobs submit --namespace --name --wait ``` **Example:** ```bash theme={null} oleander spark jobs submit process_sales_data.py --namespace finance --name process-sales-data --wait ``` Common submit options: | Flag | Description | | - | - | | `--cluster` | Cluster name. Defaults to the oleander-managed cluster when omitted | | `--namespace` | Required job namespace | | `--name` | Required job name | | `--args` | Entrypoint arguments | | `--sparkConf` | Spark configurations, for example `spark.default.parallelism=8` | | `--packages` | Extra package coordinates | | `--jobTags` | Job-specific tags | | `--runTags` | Run-specific tags | | `--wait` | Block until the job finishes | Oleander-managed submit options: | Flag | Description | | - | - | | `--driverMachineType` | Driver machine type | | `--executorMachineType` | Executor machine type | | `--executorNumbers` | Number of executor instances | ### Stop a running job Abort a run that is currently in progress: ```bash theme={null} oleander spark jobs stop ``` The run ID is returned when you submit a job and is also visible in the oleander UI. *** ## Registered clusters ### List registered clusters ```bash theme={null} oleander spark clusters list ``` ## Registered EMR Serverless Register your EMR Serverless cluster, then target it by name when submitting jobs. ### Register a cluster ```bash theme={null} oleander spark clusters register \ --type emr-serverless \ --region \ --account-id \ --controller-role-arn \ --execution-role-arn \ --application-id \ --log-bucket ``` See the [Spark documentation](/platform/compute/spark#registered-emr-serverless-spark) for IAM policy details. ### Submit a job Use an EMR-compatible entrypoint, such as an S3 URI to a `.py` or `.jar` artifact. ```bash theme={null} oleander spark jobs submit \ --cluster \ --namespace \ --name \ --wait ``` **Example:** ```bash theme={null} oleander spark jobs submit s3://my-bucket/jobs/process_sales_data.py --cluster my-emr --namespace finance --name process-sales-data --wait ``` Additional EMR Serverless flags: | Flag | Description | | - | - | | `--pyFiles` | Zip archive of Python dependencies | | `--virtualenv` | Virtual environment archive for Python jobs | | `--mainClass` | Main class for JVM jobs; use instead of Python-specific options such as `--pyFiles` and `--virtualenv` | | `--executionIamPolicy` | IAM policy applied to execution | ## Registered Glue Register your Glue cluster, then target it by name. ### Register a cluster ```bash theme={null} oleander spark clusters register \ --type glue \ --controller-role-arn ``` See the [Spark documentation](/platform/compute/spark#registered-glue-spark) for IAM policy details. ### Submit a job For Glue, the `entrypoint` is the Glue job name. ```bash theme={null} oleander spark jobs submit --cluster --namespace --name --wait ``` Additional Glue flags: | Flag | Description | | - | - | | `--workerType` | Glue worker type | | `--numberOfWorkers` | Number of Glue workers | | `--enableAutoScaling` | Enable Glue auto scaling | | `--timeoutMinutes` | Timeout in minutes | | `--executionClass` | Execution class, such as `STANDARD` or `FLEX` | | `--executionIamPolicy` | IAM policy applied to execution | For Glue jobs, `--args` must be passed as alternating key-value pairs, for example `--args source s3://bucket/in target s3://bucket/out`. # BigQuery Source: https://docs.oleander.dev/connections/bigquery Attach a Google Cloud project and query BigQuery tables alongside your Iceberg lake. Connect a Google Cloud project and every BigQuery table in it becomes queryable from the lake, joinable against your Iceberg tables in a single SQL statement. Connections are managed in [Settings → Lake](https://oleander.dev/app/settings/lake). ## Setup 1. Open [Settings → Lake](https://oleander.dev/app/settings/lake) and find **BigQuery connections**. 2. Give the connection a name - lowercase letters, numbers, and underscores, e.g. `my_bq`. This becomes the catalog prefix in SQL. 3. Paste your **service account JSON**. The GCP project ID is filled in from the key. The service account needs `roles/bigquery.dataViewer`, or narrower table-level permissions. The key is stored encrypted and is never returned to the browser after saving. ## Querying Tables are reachable as `connection_name.dataset.table`: ```sql theme={null} -- Query a BigQuery table SELECT * FROM my_bq.analytics.events WHERE _PARTITIONDATE = '2024-06-01' LIMIT 100; -- Join BigQuery with your Iceberg lake SELECT e.user_id, u.plan FROM my_bq.analytics.events e JOIN oleander.default.users u ON e.user_id = u.id WHERE e.event_type = 'signup'; ``` **A BigQuery table selects DuckDB.** DuckDB is the only engine that attaches external connections, so the [query router](/platform/query-routing/overview) routes any query naming a connection table here before it considers input size - leave `engine` on `auto`. Asking for Bloom, Polars, or Spark on one of these queries returns an engine capability error rather than rerouting silently. ## Cost attribution BigQuery spend is pulled into the context graph alongside your own runs. Use `bigquery_cost_get` from an agent, or the cost views in [observability](/platform/observability/overview), to see cost per table broken down by producing pipeline, query type, and full-scan detection. ## From an agent An agent connected over [MCP](/mcp/introduction) reaches the same tables through `query_run`. It discovers them with `catalogs_list` and queries `connection.dataset.table` directly - no BigQuery-specific tool needed. # MongoDB Source: https://docs.oleander.dev/connections/mongodb Query a live MongoDB database from the lake, and import collections into Iceberg once or on a schedule. Register a MongoDB database and its collections become a live, read-only catalog in the lake. Query them directly for ad hoc work, or import a collection into Iceberg when you want a snapshot you can run large jobs against. Connections are managed in [Settings → Lake](https://oleander.dev/app/settings/lake). Point oleander at a secondary or a dedicated read-only user, and make sure your cluster's network access list allows oleander's egress. ## Setup 1. Open [Settings → Lake](https://oleander.dev/app/settings/lake) and choose **Add MongoDB connection**. 2. Paste a `mongodb://` or `mongodb+srv://` connection string to fill the fields automatically, or enter them by hand: | Field | Description | | - | - | | **Connection name** | Lowercase letters, numbers, and underscores, e.g. `my_mongo`. This becomes the catalog prefix in SQL. Names are shared across all connection types, so it cannot match an existing Postgres, MySQL, or other connection. | | **Host** | Cluster or server host, e.g. `cluster0.abcde.mongodb.net` | | **SRV record** | Check for `mongodb+srv://` hosts, which is what Atlas uses | | **Port** | Defaults to `27017`. Not used with SRV | | **Database** | The database to expose | | **Options** | Extra URI options, e.g. `authSource=admin` | | **Username** | The user oleander connects as | | **Password** | Stored encrypted and never returned to the browser | The dialog includes the shell commands for a read-only user if you want to create one before saving. ## Querying live Collections are reachable as `connection_name.database.collection`: ```sql theme={null} -- Read straight from MongoDB SELECT _id, email, created_at FROM my_mongo.app_production.users WHERE created_at >= '2026-01-01' LIMIT 100; -- Join MongoDB against your Iceberg lake SELECT u.email, count(*) AS events FROM my_mongo.app_production.users u JOIN oleander.default.events e ON e.user_id = u._id GROUP BY 1; ``` Nested documents are flattened for SQL: a field at `address.city` is the column `address_city`. In the lake tree, field lists are sampled from the first 200 documents and `_id` is shown as the key. **A MongoDB collection selects DuckDB.** DuckDB is the only engine that attaches external connections, so the [query router](/platform/query-routing/overview) routes any query naming a connection collection there before it considers input size - leave `engine` on `auto`. Asking for Bloom, Polars, or Spark on one of these queries returns an engine capability error rather than rerouting silently. Live reads run against your database, so filter and limit the way you would against production. Once a collection is [imported](#importing-into-iceberg) it lives in Iceberg and the router is free to send it to any engine. ## Importing into Iceberg For anything larger than an ad hoc read, import the collection. The import runs as a Spark job using the MongoDB Spark connector and lands the data as an Iceberg table in your lake, where every engine can reach it. The schema is inferred from a sample of documents: * Nested documents are kept as structs and arrays as arrays, so the imported table is not flattened the way live queries are. * A field seen with conflicting types is stored as a string. A field that is only ever null is also stored as a string. * Integers and decimals are widened so later, larger values still fit. * `_id` is stored as a string, with ObjectIds as their 24-character hex. The destination defaults to `oleander.default.` - the sanitized collection name, with no connection prefix. `OVERWRITE` replaces it, `APPEND` adds to it. An import does not return rows. It submits a job and returns a `run_id` you can monitor like any other run, and once it completes you can sample the destination with a normal query. ## Collection syncing A one-off import gives you a snapshot. Put the same import on a schedule and the Iceberg copy tracks the source instead. Schedules run **hourly** or **daily**, and are managed the same way as [Postgres syncs](/connections/postgres#table-syncing): under **Scheduled syncs** in [Settings → Lake](https://oleander.dev/app/settings/lake), and under **Transfers** in the lake's left panel. **Scheduled syncs are incremental.** After the first full import, each run merges **new and changed documents** on `_id` rather than recopying the collection. How a run finds what changed depends on the collection, and the reason is written to the run's plan log: * A timestamp field such as `updatedAt`, `updated_at`, `lastModifiedAt`, or `modifiedAt` means documents modified since the last run are merged. * Otherwise, if `_id` values are ObjectIds, documents inserted since the newest one's embedded creation time are merged. Updates to existing documents are not picked up on this path. * Anything else falls back to a full refresh. New top-level fields that appear in the source are added as columns; new fields inside an existing struct wait for the next full refresh. **Deletes are never synced.** A document removed from MongoDB stays in the Iceberg copy. Every sync run is a normal oleander run: it appears in run history, emits lineage with the collection as the input and the Iceberg table as the output, and carries cost like any other job. ## From an agent Two tools cover MongoDB over [MCP](/mcp/introduction): | Tool | What it does | | - | - | | `mongo_connections_list` | List registered connections - name, host, port, SRV, database, username, options. Credentials are never returned. | | `mongo_collections_import` | Import a collection into Iceberg with the sampled-schema Spark read described above | For ad hoc reads of live data an agent uses `query_run` against `connection.database.collection` instead. ## Permissions MongoDB connections are an IAM resource type. Registering one requires **Create** on **MongoDB connections → All MongoDB connections**. Browsing a connection's collections in the lake tree and importing from it require **Describe** on that specific connection, and scheduled syncs check the same permission for the principal that runs them. See [Resources and actions](/iam/resources-and-actions#connections). # MySQL Source: https://docs.oleander.dev/connections/mysql Query a live MySQL database from the lake, and import tables into Iceberg with a parallel read once or on a schedule. Register a MySQL database and it becomes a live, read-only catalog in the lake. Query it directly for ad hoc work, or import tables into Iceberg when you want a snapshot you can run large jobs against. Connections are managed in [Settings → Lake](https://oleander.dev/app/settings/lake). For production databases, point oleander at a read replica with a dedicated read-only user, and make sure the database firewall allows oleander's egress. ## Setup 1. Open [Settings → Lake](https://oleander.dev/app/settings/lake) and choose **Add MySQL connection**. 2. Paste a `mysql://` URL to fill the fields automatically, or enter them by hand: | Field | Description | | - | - | | **Connection name** | Lowercase letters, numbers, and underscores, e.g. `my_mysql`. This becomes the catalog prefix in SQL. Names are shared across all connection types, so it cannot match an existing Postgres, MongoDB, or other connection. | | **Host** | Database host, e.g. `db.example.com` | | **Port** | Defaults to `3306` | | **Database** | The database to expose | | **SSL mode** | `DISABLED`, `PREFERRED` (default), `REQUIRED`, `VERIFY_CA`, or `VERIFY_IDENTITY` | | **Username** | The user oleander connects as | | **Password** | Stored encrypted and never returned to the browser | The dialog generates the SQL for a read-only user if you want to create one - copy it and run it against your database before saving: ```sql theme={null} CREATE USER 'oleander_reader'@'%' IDENTIFIED BY ''; GRANT SELECT, SHOW VIEW ON `app_production`.* TO 'oleander_reader'@'%'; ``` ## Querying live A connection is scoped to one database, so tables are reachable as `connection_name.database.table`: ```sql theme={null} -- Read straight from MySQL SELECT id, email, created_at FROM my_mysql.app_production.users WHERE created_at >= '2026-01-01' LIMIT 100; -- Join MySQL against your Iceberg lake SELECT u.email, count(*) AS events FROM my_mysql.app_production.users u JOIN oleander.default.events e ON e.user_id = u.id GROUP BY 1; ``` In the lake tree, column lists come from `information_schema` with primary key columns marked. **A MySQL table selects DuckDB.** DuckDB is the only engine that attaches external connections, so the [query router](/platform/query-routing/overview) routes any query naming a connection table there before it considers input size - leave `engine` on `auto`. Asking for Bloom, Polars, or Spark on one of these queries returns an engine capability error rather than rerouting silently. Live reads run against your database, so treat them the way you would treat any query against production: filter, limit, and prefer a replica. Once a table is [imported](#importing-into-iceberg) it lives in Iceberg and the router is free to send it to any engine. ## Importing into Iceberg For anything larger than an ad hoc read, import the table. The import runs as a Spark JDBC read and lands the data as an Iceberg table in your lake, where every engine can reach it. * The read is split into parallel ranges over the table's integer primary key, with the partition count following table size. A table without an integer primary key is read on a single connection. * `DATETIME` values keep their wall-clock time, and zero dates (`0000-00-00`) are read as `NULL`. Unlike [Postgres imports](/connections/postgres#importing-into-iceberg), MySQL cannot share one snapshot across connections, so a parallel import of a live, changing table is not guaranteed to be consistent across partitions. Import from a replica, or during a quiet period, when that matters. The destination defaults to `oleander.default.` - the sanitized source table name, with no connection prefix. `OVERWRITE` replaces it, `APPEND` adds to it. For large tables, prefer more executors (4-8) over bigger machines. Each executor core opens one connection to the source database. An import does not return rows. It submits a job and returns a `run_id` you can monitor like any other run, and once it completes you can sample the destination with a normal query. ## Table syncing A one-off import gives you a snapshot. Put the same import on a schedule and the Iceberg copy tracks the source instead. Schedules run **hourly** or **daily**, and are managed the same way as [Postgres syncs](/connections/postgres#table-syncing): under **Scheduled syncs** in [Settings → Lake](https://oleander.dev/app/settings/lake), and under **Transfers** in the lake's left panel. **Scheduled syncs are incremental** where the table allows it. After the first full import, each run picks up new and changed rows rather than recopying the table. What a run does depends on the table's keys, and the reason is written to the run's plan log: | Table has | Each run | | - | - | | A timestamp cursor column and a primary key | Merges rows changed since the last run | | An integer primary key only | Appends rows with a key above the last one seen | | Neither | Falls back to a full refresh | Every sync run is a normal oleander run: it appears in run history, emits lineage with the MySQL table as the input and the Iceberg table as the output, and carries cost like any other job. ## From an agent Two tools cover MySQL over [MCP](/mcp/introduction): | Tool | What it does | | - | - | | `mysql_connections_list` | List registered connections - name, host, port, database, username, SSL mode. Credentials are never returned. | | `mysql_tables_import` | Import a table into Iceberg with the partitioned, parallel read described above | For ad hoc reads of live data an agent uses `query_run` against `connection.database.table` instead. ## Permissions MySQL connections are an IAM resource type. Registering one requires **Create** on **MySQL connections → All MySQL connections**. Browsing a connection's tables in the lake tree and importing from it require **Describe** on that specific connection, and scheduled syncs check the same permission for the principal that runs them. See [Resources and actions](/iam/resources-and-actions#connections). # Postgres Source: https://docs.oleander.dev/connections/postgres Query a live Postgres database from the lake, and import tables into Iceberg with a parallel, snapshot-consistent read. Register a Postgres database and it becomes a live, read-only catalog in the lake. Query it directly for ad hoc work, or import tables into Iceberg when you want a snapshot you can run large jobs against. Connections are managed in [Settings → Lake](https://oleander.dev/app/settings/lake). For production databases, point oleander at a read replica with a dedicated read-only user. ## Setup 1. Open [Settings → Lake](https://oleander.dev/app/settings/lake) and choose **Add Postgres connection**. 2. Paste a connection string to fill the fields automatically, or enter them by hand: | Field | Description | | - | - | | **Connection name** | Lowercase letters, numbers, and underscores, e.g. `my_postgres`. This becomes the catalog prefix in SQL. | | **Host** | Database host, e.g. `db.example.com` | | **Port** | Defaults to `5432` | | **Database** | The database to expose | | **SSL mode** | Defaults to `prefer` | | **Username** | The role oleander connects as | | **Password** | Stored encrypted and never returned to the browser | The dialog generates the SQL for a read-only user if you want to create one - copy it and run it against your database before saving. ## Querying live Tables are reachable as `connection_name.schema.table`: ```sql theme={null} -- Read straight from Postgres SELECT id, email, created_at FROM my_postgres.public.users WHERE created_at >= '2026-01-01' LIMIT 100; -- Join Postgres against your Iceberg lake SELECT u.email, count(*) AS events FROM my_postgres.public.users u JOIN oleander.default.events e ON e.user_id = u.id GROUP BY 1; ``` **A Postgres table selects DuckDB.** DuckDB is the only engine that attaches external connections, so the [query router](/platform/query-routing/overview) routes any query naming a connection table here before it considers input size - leave `engine` on `auto`. Asking for Bloom, Polars, or Spark on one of these queries returns an engine capability error rather than rerouting silently. Live reads run against your database, so treat them the way you would treat any query against production: filter, limit, and prefer a replica. Once a table is [imported](#importing-into-iceberg) it lives in Iceberg and the router is free to send it to any engine. ## Importing into Iceberg For anything larger than an ad hoc read, import the table. The import runs as a Spark JDBC read and lands the data as an Iceberg table in your lake, where every engine can reach it. The import plans itself from Postgres catalog metadata: * Partition count follows table size. * Partition ranges come from the integer primary key, balanced by histogram, or from physical `ctid` chunks when there is no suitable key. * Every partition reads from **one exported MVCC snapshot**, so the copy is consistent even against a live, changing database. Reads run with a bounded number of connections against the source. It is suitable for very large tables. The destination defaults to `oleander.default.
` - the sanitized source table name, with no connection prefix. `OVERWRITE` replaces it, `APPEND` adds to it. For large tables, prefer more executors (4-8) over bigger machines. Each executor core opens one connection to the source database. An import does not return rows. It submits a job and returns a `run_id` you can monitor like any other run, and once it completes you can sample the destination with a normal query. ## Table syncing A one-off import gives you a snapshot. Put the same import on a schedule and the Iceberg copy tracks the source instead. **Scheduled syncs are incremental.** After the first full import, each run picks up **new and changed rows only** rather than recopying the table, so a sync over a large table costs a fraction of the initial import. Schedules run **hourly** or **daily**. Each one is a `connection.source_table → destination_table` pair, listed under **Scheduled syncs** in [Settings → Lake](https://oleander.dev/app/settings/lake) with its cadence and next run time, where it can be paused or removed. The same schedules also show up as **Transfers** in the lake's left panel, under the **Schedules** tab next to scheduled queries. Each row shows the source and destination tables, a health indicator from the last run, and the next run time; expand it for run history, or pause/delete it from there instead of Settings. Tables with an active sync are marked with a small lightning-bolt icon in the catalog tree. A one-time import doesn't create a schedule, so it never appears under Transfers - only recurring syncs do. Track a one-time import's progress in [observability](/platform/observability/overview) instead. Every sync run is a normal oleander run: it appears in run history, emits lineage with the Postgres table as the input and the Iceberg table as the output, and carries cost like any other job. A sync that starts failing or silently stops moving rows shows up in [observability](/platform/observability/overview) the same way a pipeline regression does. Pick a cadence against how fresh the lake copy needs to be, not how often the source changes - the incremental read is cheap, but each run is still a Spark job against your database. ## From an agent Two tools cover Postgres over [MCP](/mcp/introduction): | Tool | What it does | | - | - | | `postgres_connections_list` | List registered connections - name, host, port, database, username. Credentials are never returned. | | `postgres_tables_import` | Import a table into Iceberg with the partitioned, snapshot-consistent read described above | For ad hoc reads of live data an agent uses `query_run` against `connection.schema.table` instead. # Snowflake Source: https://docs.oleander.dev/connections/snowflake Query Snowflake tables from the lake, or register Snowflake Horizon as an Iceberg catalog. There are two ways to bring Snowflake into oleander. A **connection** attaches your Snowflake account so its tables are queryable from the lake. **Snowflake Horizon** registers Snowflake's Iceberg catalog so your Iceberg tables are available to every engine, including Spark and Polars. Both are managed in [Settings → Lake](https://oleander.dev/app/settings/lake). ## Connection Authentication uses key-pair signing. No passwords are stored. ### Setup 1. Open [Settings → Lake](https://oleander.dev/app/settings/lake) and find **Snowflake connections**. 2. Fill in the required fields: | Field | Description | | - | - | | **Name** | A short identifier for this connection, e.g. `prod_sf`. This becomes the catalog prefix in SQL. | | **Account identifier** | Your Snowflake account in `orgname-accountname` format | | **Username** | The Snowflake user to authenticate as | | **Private key** | RSA private key in PEM format. The user must have the matching public key registered in Snowflake. | 3. Optionally set a default **warehouse**, **database**, and **role** to scope queries. ### Generating a key pair ```bash theme={null} # Generate an RSA key pair openssl genrsa -out snowflake_key.pem 2048 openssl rsa -in snowflake_key.pem -pubout -out snowflake_key.pub ``` ```sql theme={null} -- Register the public key with your Snowflake user ALTER USER my_user SET RSA_PUBLIC_KEY=''; ``` Paste the contents of `snowflake_key.pem` into the **Private key** field. The key is stored encrypted. ### Querying Tables are reachable as `connection_name.schema.table`: ```sql theme={null} -- Query a Snowflake table SELECT * FROM prod_sf.public.orders WHERE created_at >= '2024-01-01' LIMIT 100; -- Join Snowflake with your Iceberg lake SELECT o.order_id, c.segment FROM prod_sf.public.orders o JOIN oleander.default.customers c ON o.customer_id = c.id; ``` **A Snowflake connection table selects DuckDB.** DuckDB is the only engine that attaches external connections, so the [query router](/platform/query-routing/overview) routes any query naming a connection table here before it considers input size - leave `engine` on `auto`. Asking for Bloom, Polars, or Spark on one of these queries returns an engine capability error rather than rerouting silently. A Horizon catalog is different: it is a real Iceberg catalog, so DuckDB and Spark can both read it. *** ## Snowflake Horizon Snowflake Horizon is Snowflake's built-in Iceberg catalog, powered by [Polaris](https://www.polaris.io/). Registering it gives you full Iceberg REST catalog access against your Snowflake-managed Iceberg tables - DuckDB queries, Polars workloads, and Spark jobs alike. Unlike a connection, a Horizon catalog is a real Iceberg catalog, so it is not limited to DuckDB. ### Setup 1. Open [Settings → Lake](https://oleander.dev/app/settings/lake) and find **External catalogs**. 2. Select **Snowflake Horizon** as the catalog type. 3. Fill in the required fields: | Field | Description | | - | - | | **Name** | A short catalog name, e.g. `sf_horizon` | | **Catalog URI** | Your Snowflake account URL, e.g. `https://orgname-accountname.snowflakecomputing.com` | | **Database** | The Snowflake database to expose as an Iceberg catalog | | **Role** | The Snowflake role used to access the catalog | | **Cloud provider** | `aws`, `gcp`, or `azure` | | **Region** | The cloud region your Snowflake account is in | 4. Choose an authentication method: Generate a PAT in Snowflake and paste it into the **Secret** field. This is the simplest option. ```sql theme={null} -- Generate a PAT in Snowflake ALTER USER my_user ADD PROGRAMMATIC ACCESS TOKEN my_pat; ``` Use an RSA key pair for authentication. Provide: | Field | Description | | - | - | | **Username** | The Snowflake user | | **Private key** | RSA private key in PEM format | | **Passphrase** | Passphrase for the private key, if encrypted | The same key pair used for a Snowflake connection can be reused here. ### Querying Once registered, the catalog is available under its name everywhere: ```sql theme={null} -- DuckDB: query a Snowflake Horizon Iceberg table SELECT species, COUNT(*) AS n FROM sf_horizon.default.flowers GROUP BY 1 ORDER BY n DESC; ``` ```python theme={null} # Polars script mode flowers = scan("sf_horizon.default.flowers") result = flowers.group_by("species").agg(pl.len().alias("count")) ``` Credential vending is handled automatically via the Iceberg REST protocol. No S3 or GCS credentials need to be configured manually. A catalog other than `oleander` routes to DuckDB or Spark - [Bloom](/platform/compute/bloom) mounts only `oleander`. See [Catalogs](/platform/storage/catalogs) for how registered catalogs behave across the platform. # Overview Source: https://docs.oleander.dev/iam/overview Control who can use compute, access data, and administer your organization. Identity and access management (IAM) connects **who is acting** to **what they may do**. Permissions and role assignments are scoped to an organization; access in one organization does not grant access in another. ## How access fits together ```mermaid theme={null} flowchart TD accTitle: IAM relationships accDescr: An API key authenticates as its owning principal. Principals receive roles containing permissions that allow actions on resources within a scope. K[API key] -->|authenticates as its owner| P[Principal] P -->|is assigned| R[Roles] R -->|contain| G[Permissions] G -->|allow actions within a scope| T[Resources] ``` | Component | Purpose | | - | - | | [Principal](/iam/principals) | A Human, Application, or Agent identity that acts in the organization. | | [Role](/iam/roles) | A named collection of permissions assigned to one or more principals. | | [Permission](/iam/permissions) | An allowed action on a resource within a scope. | | [API key](/platform/settings/api-keys) | A credential owned by one principal. Its IAM access comes from that principal's roles. | A principal can hold several roles, and their grants combine. Only **Allow** permissions are supported: a role with fewer permissions does not cancel access granted by another role. Creating a permission grants no access until it belongs to an assigned role. For an IAM-controlled action, the required grant must match the action and resource scope. Without it, access is denied. ## Compute and data access Compute permissions control whether a principal can run or abort work on an engine. Warehouse permissions control which data the principal can read or change. Running a query against the warehouse needs both appropriate compute and data access. Catalog and connection permissions separately control discovery and management of registrations. They do not grant access inside an external database. Start with [Roles](/iam/roles) to choose a built-in role or create a custom one. Use [Resources and actions](/iam/resources-and-actions) for the complete permission reference. # Permissions Source: https://docs.oleander.dev/iam/permissions Define allowed actions, choose their scope, and delegate access through roles. A permission allows one action on a resource within a scope. Add permissions when creating a role or through its **Permissions** action in [Settings → Organization → Roles](https://oleander.dev/app/settings/roles). | Field | Meaning | Example | | - | - | - | | Resource type | The kind of resource being controlled. | Compute | | Scope | Which resources the permission applies to. | Selected resources: `duckdb` | | Actions | What the principal may do. | Execute | Selecting multiple resources and actions creates a permission for each resource/action combination. Every permission is **Allow**; explicit Deny permissions are not supported. A definition can be shared by several roles, but has no effect until attached to a role assigned to a principal. ## Choosing a scope Broad scopes such as **All engines**, **Entire warehouse**, and **All application principals** include matching resources created later. **Selected resources** refers to the chosen objects; **Selected namespaces** selects warehouse namespaces. **Self (whoever holds this role)** means the principal making the request, including an API key's owner. It does not mean the role's creator. For **API keys**, selecting a principal scopes the permission to that principal's keys. The available actions change with the scope. For example, creating an Application principal requires **All application principals**; a selected Application principal can be deleted but is already created. Available resource choices also depend on the caller's discovery access. ## Delegating access **Manage grants** permits delegation within its scope, including delegation of **Manage grants** itself. It does not automatically grant operational actions such as **Execute** or **Describe**. | Operation | Required authority | | - | - | | Assign or unassign a role | Manage grants on **both** the target principal and role. | | Attach or detach a permission | Manage grants on **both** the target role and the resource the permission controls. | | Invite a Human with roles | Invite for All human principals, plus Manage grants on each selected role. | | Configure another execution principal | Describe and Manage grants on that principal, plus its eligibility for the requested operation. | Permission definitions are themselves manageable resources. **Manage grants** on a permission definition does not substitute for authority over the compute engine, warehouse, or other resource it describes. A grant for selected resources cannot delegate authority over all resources. Oleander's namespace delegation check matches the selected namespace exactly, or **Entire warehouse**: authority to attach permissions for `finance` does not authorize attaching permissions for `finance.staging`. Warehouse **Pass grants** permits delegation of held warehouse privileges, excluding Pass grants itself. It does not replace the Manage grants requirements for attaching permissions to oleander roles. Warehouse data inheritance is explained in the [reference](/iam/resources-and-actions#warehouse). ## Example: query one namespace These permissions combine compute execution with read access to an existing namespace: | Resource type | Scope | Actions | | - | - | - | | Compute | Selected resources: `duckdb` | Execute | | Warehouse | Selected namespaces: `finance` | Select | Attach both to a custom role and assign the role to the intended principal. They allow DuckDB execution and reads within `finance`; they grant neither data modification nor job cancellation. Other assigned roles may grant additional access. Catalog discovery and credential retrieval have separate **Catalogs → Describe** requirements. Add those grants when the workflow needs them; compute execution alone does not supply them. Use [Resources and actions](/iam/resources-and-actions) to choose the precise scope and actions for each permission. # Principals Source: https://docs.oleander.dev/iam/principals Human, Application, and Agent identities, including the system-managed OleanderSystem principal. A principal is an identity that can receive [roles](/iam/roles) and own [API keys](/platform/settings/api-keys). Manage principals in [Settings → Organization → Users](https://oleander.dev/app/settings/members), under the **Human**, **Application**, and **Agent** tabs. ## Principal types | Type | Used for | Lifecycle | | - | - | - | | Human | People who sign in to oleander. | Invite to the organization; remove their membership when access is no longer needed. | | Application | Services, integrations, and automated workloads. | Create an identity in the organization; delete it when no longer needed. | | Agent | AI agents that need their own access. | Create an identity in the organization; delete it when no longer needed. | Removing a Human removes membership in the current organization, not the person's account. Deleting an Application or Agent deletes that organization-owned identity. Organization owners and system-managed principals are protected from these removal operations. New custom Application and Agent principals receive a default API key, but no roles automatically. Assign the roles they need before using IAM-managed resources. Human invitations can include roles; inviting requires **Invite** for **All human principals** and **Manage grants** on each selected role. ## Acting as a principal An API key acts as its owner. A scheduled job uses its configured execution principal, whose current organization membership and compute access are checked when it runs. A schedule can run only as the person creating it or as an Application principal; other Humans and Agents cannot be selected, and choosing an Application requires **Describe** and **Manage grants** on it. An MCP OAuth connection can act as the signed-in user or a selected, non-system-managed Agent principal. Selecting another execution principal requires **Describe** and **Manage grants** on that principal. MCP delegation is checked again when the connection refreshes its credentials. The selected principal's permissions govern the work. The person configuring the connection or schedule does not transfer their own permissions to it. ## OleanderSystem Oleander provisions an Application principal named **OleanderSystem** for each organization. It starts with the **JobOperator** role and a default API key. Internal workflows use this key for operations including OpenLineage reporting. **JobOperator** grants **Execute** and **Abort** across all compute engines; it does not grant warehouse data access. Being system-managed does not bypass IAM checks. OleanderSystem cannot be deleted. Its role assignments remain subject to the ordinary [assignment rules](/iam/permissions#delegating-access), so authorized administrators can change them. Revoking OleanderSystem's default API key can interrupt workflows that depend on it. Retrying organization provisioning preserves the existing key and its revocation state; it does not reactivate a revoked key. See [Resources and actions](/iam/resources-and-actions#principals) for the actions available at each principal scope, including currently unimplemented lifecycle actions. # Resources and actions Source: https://docs.oleander.dev/iam/resources-and-actions The complete reference for resource types, scopes, and permission actions. These tables use the **Resource type**, **Scope**, and **Actions** labels from the role permission editor. All scopes belong to the current organization. Broad scopes include matching future resources; selected scopes apply to the chosen objects. Only **Allow** permissions are supported. An action listed here is a valid permission definition; limitations below identify operations that are not implemented. See [Permissions](/iam/permissions) for assignment and delegation rules. ## Compute | Resource type | Scope | Actions | | - | - | - | | Compute | All engines | Execute, Abort, Manage grants | | Compute | Selected resources: `spark`, `duckdb`, `bloom`, or `polars` | Execute, Abort, Manage grants | **Execute** submits queries or jobs to the engine. **Abort** cancels work on the engine. **Manage grants** delegates these permissions, including grant-management authority. Compute grants do not provide warehouse data access. ## Warehouse | Resource type | Scope | Actions | | - | - | - | | Warehouse | Entire warehouse | Describe, Select, Create, Modify, Manage grants, Pass grants | | Warehouse | Selected namespaces | Describe, Select, Create, Modify, Manage grants, Pass grants | These permissions control data in the organization's built-in warehouse. Namespace names can be nested, such as `finance.staging`. | Action | Meaning within the scope | | - | - | | Describe | Discover objects and read their metadata. | | Select | Read table and view data. | | Create | Create child namespaces, tables, and views. | | Modify | Change object contents or properties. | | Manage grants | Administer grants, including Manage grants and Pass grants. | | Pass grants | Delegate held privileges, but not Pass grants itself. | Warehouse access is enforced by the Iceberg catalog, Lakekeeper. Data grants inherit down the warehouse/namespace hierarchy. **Select** and **Create** include **Describe**; **Modify** includes **Select** and **Describe**. These implications apply to warehouse data access, not to action matching for other oleander resource types. Attaching warehouse permissions to an assigned role makes them effective in the catalog. **Pass grants** does not replace oleander's requirement for **Manage grants** on both the role and the permission's resource when changing a role's permissions. ### Reserved table and view scopes Exact table and view permissions are reserved and cannot currently be created or attached to roles. They are absent from the UI chooser; use namespace permissions instead. | Resource type | Reserved scope | Defined actions | | - | - | - | | Warehouse table | One named table in one named namespace | Describe, Select, Modify, Manage grants, Pass grants | | Warehouse view | One named view in one named namespace | Describe, Select, Modify, Manage grants, Pass grants | These reserved scopes require exact namespace and object names. There is no all-tables or all-views scope, and Create is not defined on an existing table or view. ## Catalogs | Resource type | Scope | Actions | | - | - | - | | Catalogs | All catalogs | Describe, Create, Modify, Delete, Manage grants | | Catalogs | Selected resources | Describe, Modify, Delete, Manage grants | These permissions manage Iceberg catalog registrations, including the built-in `oleander` catalog and registered external catalogs. | Action | Meaning | | - | - | | Describe | View catalog information and retrieve catalog credentials. | | Create | Register a catalog. | | Modify | Update a catalog's registration configuration. | | Delete | Remove a registration, not its warehouse data. | | Manage grants | Delegate catalog-registration permissions. | Catalog listing currently requires **Describe → All catalogs**; a grant for one selected catalog does not authorize the full list. Credential retrieval also requires current organization membership. Catalog-registration grants do not themselves grant access to its data. The built-in warehouse uses warehouse permissions; external catalogs enforce their own data access. The `oleander` catalog cannot be deleted, even by **OrganizationAdmin**. ## Connections | Resource type | Scope | Actions | | - | - | - | | Postgres connections | All Postgres connections | Describe, Create, Delete, Manage grants | | Postgres connections | Selected resources | Describe, Delete, Manage grants | | BigQuery connections | All BigQuery connections | Describe, Create, Delete, Manage grants | | BigQuery connections | Selected resources | Describe, Delete, Manage grants | | Snowflake connections | All Snowflake connections | Describe, Create, Delete, Manage grants | | Snowflake connections | Selected resources | Describe, Delete, Manage grants | | MongoDB connections | All MongoDB connections | Describe, Create, Delete, Manage grants | | MongoDB connections | Selected resources | Describe, Delete, Manage grants | | MySQL connections | All MySQL connections | Describe, Create, Delete, Manage grants | | MySQL connections | Selected resources | Describe, Delete, Manage grants | **Describe** discovers connection registrations. **Create** registers a connection. **Delete** removes its registration, not the external database. **Manage grants** delegates these permissions. There is no Modify action or single scope covering all connection types. Connection discovery does not grant permissions inside the external database or blanket access to stored passwords. Importing data also requires the appropriate compute and destination-data permissions. For [MongoDB](/connections/mongodb) and [MySQL](/connections/mysql), browsing a connection in the lake tree and importing from it also require **Describe** on that exact connection, and scheduled syncs check it for the principal that runs them. ## Audit logs | Resource type | Scope | Actions | | - | - | - | | Audit logs | All audit logs | Describe, Manage grants | Only the organization-wide scope exists; there are no per-entry permissions and no **Selected resources** option. | Action | Meaning | | - | - | | Describe | View the [audit log](/platform/settings/audit-logs), including principal identities. Does not require permissions on the resources the entries mention. | | Manage grants | Delegate audit log permissions. Does not itself grant viewing. | Recording is internal, so there are no Create, Modify, or Delete actions. `OrganizationAdmin` holds both actions, `SecurityAdmin` holds Describe, and `UserAdmin` holds Manage grants. Enforcement of Describe on the audit page follows once system-managed roles have been backfilled for every organization. ## Principals | Resource type | Scope | Actions | | - | - | - | | Principals | All principals | Describe, Modify, Enable, Disable, Manage grants | | Principals | Self (whoever holds this role) | Describe, Modify, Enable, Disable, Manage grants | | Principals | All human principals | Describe, Modify, Enable, Disable, Manage grants, Invite, Remove | | Principals | Selected resources: Human principals | Describe, Modify, Enable, Disable, Manage grants, Remove | | Principals | All application principals | Describe, Modify, Enable, Disable, Manage grants, Create, Delete | | Principals | Selected resources: Application principals | Describe, Modify, Enable, Disable, Manage grants, Delete | | Principals | All agent principals | Describe, Modify, Enable, Disable, Manage grants, Create, Delete | | Principals | Selected resources: Agent principals | Describe, Modify, Enable, Disable, Manage grants, Delete | | Action | Meaning | | - | - | | Describe | View principal information. Principal lists include only principals the caller can describe. | | Modify | Defined for editing principal details; the operation is not implemented yet. | | Enable / Disable | Defined for activation changes; the operations are not implemented yet. | | Manage grants | Manage role assignments, also requiring Manage grants on the role. Separate Describe and Manage grants checks apply when delegating execution to a principal. | | Invite | Invite a Human to the organization; roles on the invitation also require Manage grants on each role. | | Remove | Remove a Human's organization membership, preserving their account. | | Create | Create an Application or Agent principal. | | Delete | Delete an Application or Agent principal. System-managed principals are protected. | Use the appropriate type-specific scope for Invite, Remove, Create, and Delete. **All principals** and **Self (whoever holds this role)** do not support those lifecycle actions. Principal permissions do not grant access to the principal's API keys. ## API keys The Scope field selects **key owners**. In this category, **Selected resources** selects principals and applies to all their keys; it does not select individual keys. | Resource type | Scope | Actions | | - | - | - | | API keys | All principals | Describe, Create, Revoke, Manage grants | | API keys | Self (whoever holds this role) | Describe, Create, Revoke, Manage grants | | API keys | All human principals | Describe, Create, Revoke, Manage grants | | API keys | All application principals | Describe, Create, Revoke, Manage grants | | API keys | All agent principals | Describe, Create, Revoke, Manage grants | | API keys | Selected resources: principal owners | Describe, Create, Revoke, Manage grants | An individual key can also be targeted through the API, although the UI chooser does not expose that scope. It supports **Describe**, **Revoke**, and **Manage grants**, but not **Create**. An individual key must have a selected owner or Self; it cannot be paired with all owners or an entire principal type. | Action | Meaning | | - | - | | Describe | List an owner's keys, including their stored values. Treat this as credential access. | | Create | Issue a key owned by the selected principal. | | Revoke | Revoke a key owned by the selected principal. | | Manage grants | Delegate key-management permissions; does not itself grant Describe, Create, or Revoke. | Human keys remain self-service even with broad grants. Application and Agent keys can be managed by authorized callers. Listing an owner's keys requires Describe for that owner's key collection; an individual-key grant does not authorize listing all their keys. [Legacy keys](/platform/settings/api-keys#legacy-keys) use separate management rules. ## Roles | Resource type | Scope | Actions | | - | - | - | | Roles | All roles | Describe, Create, Delete, Manage grants | | Roles | Selected resources | Describe, Delete, Manage grants | **Describe** reads the role and its permissions and assigned principals. Role lists include only roles the caller can describe; viewing a role's assigned principals does not require separate Describe grants on each principal. **Create** creates a role definition. **Delete** removes an eligible custom role, not its principals or shared permission definitions. **Manage grants** controls role assignments and permission attachments, subject to the [second target check](/iam/permissions#delegating-access). [System-managed roles](/iam/roles#system-managed-roles) cannot be deleted or have their permission composition changed. Roles referenced by principal assignments or invitations cannot be deleted. ## Permissions | Resource type | Scope | Actions | | - | - | - | | Permissions | All permissions | Describe, Create, Delete, Manage grants | | Permissions | Selected resources | Describe, Delete, Manage grants | This resource type controls **permission definitions**, separately from the resources those definitions grant access to. **Describe** reads definitions. Listing all definitions requires **Describe → All permissions**. **Create** stores a valid permission; it does not attach it to a role. **Delete** removes an unused custom definition. System-managed definitions and definitions attached to roles cannot be deleted. **Manage grants** delegates authority over a permission definition. It does not authorize attaching that definition's compute, warehouse, or other underlying grant to a role; that requires Manage grants on both the role and the underlying resource. # Roles Source: https://docs.oleander.dev/iam/roles Group permissions into custom roles or use oleander’s system-managed roles. A role is a named collection of permissions. It can combine compute, warehouse, and administration permissions. A principal can hold multiple roles, and a role can be assigned to multiple principals. API keys use their owner's roles and cannot receive assignments directly. Manage roles in [Settings → Organization → Roles](https://oleander.dev/app/settings/roles). ## Custom roles 1. Choose **Create role**, enter a name and optional description, and add permissions. 2. For each permission, choose **Resource type**, **Scope**, and **Actions**, then **Apply permissions**. 3. Create the role and use its **Assign users** action to assign principals. Use **Permissions** to review or change its grants. Creating an empty role requires **Create** for **Roles → All roles**. Adding new permission definitions also requires **Create** for **Permissions → All permissions**. Attaching permissions and assigning principals require the additional [delegation grants](/iam/permissions#delegating-access). Changing a role's permissions affects every principal assigned that role. Removing an assignment removes that role's contribution to the principal's access; other assigned roles may still grant the same access. ## System-managed roles Oleander provisions the following roles for each organization and controls their definitions. You cannot delete them or change their permission composition. Authorized administrators can assign and unassign them. Their system-managed permission definitions cannot be edited or deleted. | Role | Grants | | - | - | | `OrganizationAdmin` | Includes all grants from the other system-managed roles, plus Describe, Select, Create, Modify, Manage grants, and Pass grants across the entire warehouse. | | `UserAdmin` | Manages principals, roles, and permission definitions; manages Application and Agent API keys; delegates access across all resource categories, including audit logs. Includes principal Modify, Enable, and Disable definitions, whose operations are not implemented yet. | | `SecurityAdmin` | Manages catalog registrations, connections, roles, and permission definitions. Delegates catalog and connection access. Views the [audit log](/platform/settings/audit-logs). Does not itself grant principal administration, compute execution, or warehouse data access. | | `JobOperator` | Execute and Abort on all compute engines. No warehouse data grants. | | `TokenSelfAdmin` | Describe, Create, and Revoke for API keys owned by Self—the principal using the role. | | `Public` | Contains no permissions. Grants no compute or data access by itself. | `UserAdmin` grants authority to delegate compute and warehouse access, without directly granting execution or data-reading actions. `SecurityAdmin` can manage role definitions, but assigning those roles still requires authority over the target principal. `OrganizationAdmin` remains subject to resource safeguards: it cannot delete system-managed definitions, delete the built-in `oleander` catalog, or manage another Human's personal API keys. ## Removing roles A custom role cannot be deleted while referenced by principal assignments or invitations. Deleting an eligible role removes its permission associations, not the shared permission definitions or principals. Unassigning the last remaining **OrganizationAdmin** is blocked. This protection applies to role unassignment; membership removal and leaving an organization do not provide the same complete last-admin protection. See [Resources and actions](/iam/resources-and-actions#roles) for role-management permissions. # Airflow Source: https://docs.oleander.dev/integrations/airflow Configure oleander with Apache Airflow ## Overview The OpenLineage integration is now contributed directly to Airflow. A comprehensive guide on how to configure and use the OpenLineage integration with Airflow can be found in [*Using OpenLineage Integration*](https://airflow.apache.org/docs/apache-airflow-providers-openlineage/stable/guides/user.html).