Skip to main content
Hero Light

Spark

Run your PySpark applications on managed Spark infrastructure. Upload scripts, execute jobs, and automatically capture lineage metadata for complete observability of your data transformations.
# process toxic flower data at massive scale:
oleander spark job submit analyze_toxic_flowers --wait
Learn more about Spark →

Lake

Run SQL queries, sync data, and collaborate on both public and private tables. Every operation automatically captures lineage metadata for observability. That lake is compatible with the same datasets used by oleander managed spark.
--
-- 🌸 flower power query 🌸
-- oleander is the deadliest flower... but hey, at least it's pretty!
SELECT
  flower_name,
  toxicity_level,
  CASE flower_name
    WHEN 'oleander' THEN '💀 DEADLIEST FLOWER ON EARTH 💀'
    WHEN 'belladonna' THEN '👀 Pretty but hallucinogenic'
    WHEN 'foxglove' THEN '💔 Beautiful heart-stopper'
    WHEN 'datura' THEN '🤯 Mind-bending madness'
    ELSE '🌺 Just regular toxic beauty'
  END AS flower_personality,
  CASE flower_name
    WHEN 'oleander' THEN '🌺 "I''m rooting for you!" 🌺'
    WHEN 'belladonna' THEN '🌸 "Don''t belladonna your eyes!" 🌸'
    WHEN 'foxglove' THEN '🦊 "Fox news: Flowers plotting!" 🦊'
    WHEN 'datura' THEN '🌵 "Datura be kidding me!" 🌵'
    ELSE '💐 Generic flower pun 💐'
  END AS flower_pun
FROM (
  VALUES
    ('oleander', 9.5),
    ('belladonna', 8.7),
    ('foxglove', 7.3),
    ('datura', 6.9),
    ('aconite', 8.1),
    ('hemlock', 7.8)
) AS toxic_flowers(flower_name, toxicity_level)
ORDER BY toxicity_level DESC;
Learn more about the lake →

Data observability

Get comprehensive insights into your data pipelines with seamless integration to Spark (oleander, or not!), Airflow, dbt, and Flink. Visualize lineage, track dependencies, and monitor execution. Learn more about observability →

Educational resources

Explore our free tools to help you work with data and OpenLineage:
  • Parquet viewer - View and inspect Parquet files directly in your browser
  • OpenLineage validator - Validate OpenLineage facets and ensure your metadata conforms to the specification
  • OpenLineage graph - Visualize and explore OpenLineage lineage graphs interactively
Graph Light