What is oleander?
oleander is your agent’s data platform team. We host and manage storage and compute tools in a unified platform with automatic query routing, so your agent can query, transform, and run jobs without managing engines or infrastructure.Who we are
oleander was founded by the co-creators of OpenLineage and Marquez. An open standard the data industry uses to emit and track lineage.The query routing layer
Send a query without naming an engine and the query routing layer decides where it runs. It parses the statement, resolves the tables it touches, and picks an engine from what the query needs: a connection or a foreign catalog means DuckDB, Iceberg DDL means Spark, a DataFrame script means Polars. Only once nothing forces a choice does size decide - estimated from Iceberg snapshot metadata, and used to pick the machine as well as the engine. Every decision carries a reason trail, returned asengine_decision, so a run is never a black box. An agent can explain exactly why a query ran where it did, and explain: true returns the whole decision without spending any compute.
Compute scoped to the request
Each query gets its own isolated sandbox, sized from that query’s estimated input rather than from an org-wide setting. A small read lands on 2 vCPUs. The same query a year later, against a table that grew tenfold, lands on a bigger one. Past 50 GiB it stops being a sandbox at all and goes distributed across sized workers. For an agent this matters more than it does for a person: it can fire off an exploratory query without first deciding what it will cost, and a query that turns out to be expensive gets the machine it needs instead of timing out on a shared one. Nothing is provisioned ahead of time and nothing stays running between requests.Beyond SQL
Routing covers queries. When the work is ingest, streaming, or code that SQL cannot express, Serverless Spark runs PySpark applications and JARs on managed infrastructure or your own cluster - JDBC and API ingest, Structured Streaming, ML training, anything with control flow. Polars scripts sit in between: real Python against your lake tables, no job to package. Both emit the same OpenLineage events as a routed query, so a Spark job and a one-off SELECT are the same kind of node in the graph. See Query routing for the rules in order.Agent in the loop
oleander is built around the idea that agents should be first-class participants in your data engineering workflow - not an afterthought bolted onto a dashboard.
“What’s the average petal length in the iris dataset, broken down by species?”The agent calls
catalogs_list to find your tables, then query_run to run the query - no SQL written by you, no engine specified, no dashboard opened.
“Build a Spark ML classifier that readsThe agent callsoleander.default.iris, uses the sepal and petal measurements to predict flower species, trains and evaluates a random forest model, and writes the test-set predictions tooleander.default.iris_predictions.”
spark_artifacts_upload to store the script, then spark_jobs_submit to run it. Once the run completes, it verifies the output automatically - comparing row counts and byte volumes against the previous window, checking the schema diff, confirming lineage, and benchmarking cost per record. If anything regresses, it opens an investigation and surfaces a fix before you’ve noticed anything.
Spark Streaming jobs follow the same pattern. The agent writes the streaming job, submits it to a connected cluster, and monitors run events as they arrive. If throughput drops or a batch fails, it surfaces the issue with full context so the engineer can act immediately.
Self-validating by default
Every run writes lineage, traces, and cost events to the lake automatically. oleander compares each run against the previous window, flags regressions in row counts, byte volumes, and duration, and surfaces anomalies before they reach production. Engineers and agents see the same signal. When the agent catches something, it opens an investigation with root cause, evidence from telemetry, and a suggested next step. No human has to notice the alert and open four tabs to figure out what happened.Multi-engine, cost-aware
Agents routing through the MCP inherit this logic. A query that fits in a sandbox never starts a cluster. A job that needs distributed compute gets one sized to its input. Cost per record is tracked on every run.
Beyond the managed engines, attach the data you already have as connections - BigQuery, Snowflake, and Postgres, queryable alongside your lake - and point the compute you already run at oleander: your own Spark, Airflow, and dbt.
How oleander works
1
Load your data
Upload files directly, sync from S3, import from Postgres, or attach BigQuery and Snowflake. Your data lands in the oleander lake as Iceberg tables, queryable immediately.
2
Connect your agent
Add the MCP server in one command. Your agent gets access to 41 tools: routed queries, catalog management, lineage tracing, run inspection, cost attribution, and Spark job submission.
3
Run and validate automatically
Every job run emits lineage and telemetry. oleander validates each run against historical baselines, flags anomalies, and triggers investigations when something looks wrong.
4
Query from anywhere
CLI, API, MCP, tasks, and SDK all read from the same lake and context graph. Engineers and agents work with the same data, the same lineage, and the same cost signal.