> ## Documentation Index
> Fetch the complete documentation index at: https://docs.oleander.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# Bloom

> oleander's own SQL engine: Apache DataFusion over Iceberg, running local on a sandbox or distributed across workers.

Bloom is the engine oleander reaches for first. It is built on [Apache DataFusion](https://datafusion.apache.org/) and [DataFusion Distributed](https://github.com/datafusion-contrib/datafusion-distributed), reads Iceberg through the same REST catalog as everything else, and runs in two shapes from the same binary: **local**, inside a single sandbox, and **distributed**, across a fleet of workers.

You do not choose between them. The [query router](/platform/query-routing) sizes the run from the input and picks the shape.

## Why DataFusion

DataFusion is a Rust query engine with a vectorized Arrow execution core, a full SQL frontend, and a physical planner that parallelizes across cores. Bloom adds three things to it:

* **An Iceberg bridge.** A DataFusion `CatalogProvider` over Iceberg REST that resolves only the tables a query actually names, rather than loading every table in every catalog.
* **Predicate pushdown into Iceberg scans.** Supported primitive predicates are pushed down so Iceberg can prune by partition, manifest, file statistics, and reader. DataFusion still re-evaluates the original filters, so pruning never changes results.
* **Distributed execution.** The coordinator plans file tasks once and streams versioned tasks to workers. Workers never contact or preload the REST catalog.

## Local mode

A local Bloom run executes entirely inside one sandbox and returns rows on the call. This is the default for interactive reads against the `oleander` catalog.

Sandbox size comes from the router's input estimate - 2 vCPU for small queries up to 32 vCPU for inputs approaching 50 GiB. See [machine sizing](/platform/query-routing#machine-sizing).

```sql theme={null}
SELECT species, count(*) AS n
FROM oleander.default.flowers
GROUP BY 1
ORDER BY n DESC;
```

## Distributed mode

Past 50 GiB of estimated input, Bloom goes distributed: a coordinator plus 2 to 10 workers, sized from the input.

| Input size     | Worker machine | Worker count           |
| -------------- | -------------- | ---------------------- |
| under 250 GiB  | `bloom.2.b`    | `ceil(bytes / 32 GiB)` |
| 250 GiB and up | `bloom.4.b`    | `ceil(bytes / 64 GiB)` |

A distributed run does not return rows. It always commits to a destination table, takes `OVERWRITE` (default) or `APPEND`, and emits lineage from the workflow once it finishes, with the destination as the output dataset.

The final worker stage writes Parquet directly and returns only Iceberg `DataFile` metadata to the coordinator, which performs one Iceberg commit. Workers use refreshable, prefix-scoped credential leases for both reads and writes, and never receive the catalog token.

<Note>
  Bloom is metered compute and requires a payment method on file. An org without one falls back to DuckDB.
</Note>

## URL tables

Bloom can read a single CSV, JSON, or Parquet file directly from an absolute URL, quoted in place of a table name:

```sql theme={null}
SELECT * FROM 'https://data.seattle.gov/resource/kzjm-xkqj.csv';
```

Anonymous `http://` and `https://` reads work in both modes. Range-capable servers are read by file-scan range; a server that ignores byte ranges or reports no length is downloaded once per runtime and cached in memory.

This is deliberately narrow. It rejects credentials, headers, query strings, fragments, directories, globs, relative paths, and cloud schemes such as `s3://`, `gs://`, and `azure://`. Use a [catalog](/platform/lake/catalogs) or an [external connection](/platform/connections/bigquery) for anything beyond one public file.

## Writing to a table

When Bloom is asked to produce a table, the write is planned as a DataFusion DML plan whose target is the final table, and the plan is exposed before physical execution so lineage is extracted from the real plan rather than from a re-parse of the SQL.

`OVERWRITE` completely replaces the destination - schema, partitioning, properties, history, and data. The replacement table is unpartitioned. `APPEND` adds to an existing compatible table, creating it unpartitioned from the result schema if it does not exist.

## Versions

| Component              | Version |
| ---------------------- | ------- |
| DataFusion             | 54      |
| DataFusion Distributed | 2.0     |
| Apache Iceberg (Rust)  | 0.9.1   |
| Arrow                  | 58      |

Iceberg 0.9.1 exposes no full-table overwrite transaction, so `OVERWRITE` is implemented by writing a replacement table and swapping it in with catalog renames after the write succeeds. The logical plan still names only the final destination.