Skip to main content

Overview

Tables in oleander live inside your private oleander Iceberg catalog. There are two namespaces:
  • oleander.default: tables you create, upload, or sync from S3
  • oleander.telemetry: platform-managed tables written automatically as your pipelines run (run_events, traces, logs)
All tables are Iceberg format, which means they support time-travel queries, schema evolution, and are readable from any Iceberg-compatible engine including Spark.

User tables (default namespace)

Tables in oleander.default are yours to create and manage. You can:
  • Run a CREATE TABLE statement in the lake SQL editor
  • Create an empty table from the catalog tree: choose Create empty table from a namespace’s more-options menu to define columns, an optional table description and column docs, and a partition spec, or edit the generated SQL directly
  • Upload a Parquet, CSV, or JSON file from the lake. See Upload files
  • Schedule an S3 sync to automatically import parquet files from your S3 bucket on a daily cadence
  • Write via DuckDB import from any external source
Every table and column in the lake’s catalog tree has a more-options menu. View statistics on a table profiles every column in one query - row, null, and distinct counts plus min and max - and on a column profiles just that one. View context graph opens the table or column in the context panel.
See Time travel for what Bloom accepts.

Table health and maintenance

Every table in oleander.default shows a health status dot at the end of its row in the catalog tree: green for healthy, amber for warning, red for critical, or grey for unknown - a writer that recorded no snapshot statistics, so there’s nothing to grade. Hover the dot for row and delete-file counts, size, and (when degraded) what’s driving the grade. The popover also shows the table’s Layout - its partition spec and sort order, with a suggested partition column when a large table has none - and its Freshness - the last commit, and for tables written by Streamkap or Kafka Connect, the writer and the time the table is complete through. The usual cause is delete-file pressure: a CDC sink or streaming writer (Debezium, Kafka Connect, Flink, Fivetran) issues equality or position deletes that accumulate between compactions. A table with heavy delete pressure is routed to Spark for queries rather than DuckDB or Bloom, since applying a large equality-delete anti-join is what makes those queries slow to the point of not returning rather than just slow - explicitly requesting a different engine still runs it there, with a warning, instead of silently overriding your choice. Equality deletes written before any data they could match - the first commit of an upsert-mode CDC sink, for example - remove no rows, so they’re left out of the grade and don’t send the table to Spark. The popover says how many there are; maintenance drops them. Click the dot to schedule maintenance - it reads Schedule maintenance on a healthy or ungradeable table and Run maintenance on one that needs it. Maintenance compacts the table on Spark, rewriting data files and clearing the delete files that rewrite strands. Runs can fire once or on a recurring schedule; scheduled maintenance is listed under Maintenance in the lake panel’s Schedules tab, next to scheduled queries and transfers, with pause/resume/delete, per-operation toggles, a Run as… entry to change the execution principal, and run history. Every run appears on the Workloads page like any other job and is billed as Spark compute. The dialog’s Partitioning tab changes how a table is laid out. Build the new layout from the table’s columns - by hour, day, month or year of a timestamp, each distinct value, hash buckets, or ranges of a number or the first characters of a string - or click Suggest with AI to have oleander weigh the table’s size against the columns your recent queries filter on and propose a spec, keep the current one, or say the table is too small to be worth partitioning. The tab lists the table’s current partitions with their files, rows, and sizes, and shows a live estimate of how many partitions the edited layout would produce and how large they would be. With Rewrite existing data now checked (the default), Repartition now changes the partition spec and runs once on Spark as the chosen execution principal to rewrite every data file into the new layout and expire old snapshots. Uncheck it and Change spec only changes the spec, instantly and at no cost: new writes use the new layout and existing files keep theirs. Repartitioning can’t be scheduled. A large table with no partition spec, or one with about one small file per partition that compaction can’t merge, is graded on its layout: the health popover says what to try, and its button reads Repartition and opens the dialog on this tab.

Telemetry tables (telemetry namespace)

The telemetry namespace is populated automatically. These tables capture everything that flows through oleander’s observability pipeline - you can query them like any other table.
See Catalogs for full schema definitions of the telemetry tables.

Querying across sources

Because BigQuery, Snowflake, and Postgres connections and external Iceberg catalogs are attached to the same DuckDB session, you can join across all of them in a single query:
See BigQuery, Snowflake, and Postgres for how to set up connections, and Catalogs for external Iceberg catalogs. A query that names a connection table always runs on DuckDB.