> ## Documentation Index
> Fetch the complete documentation index at: https://docs.oleander.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# Spark SQL

> Submit Spark SQL queries that write results to Iceberg tables.

## `submit_spark_sql(options)`

Submit a Spark SQL query that writes its result to an Iceberg output table. Returns run info immediately; use `get_run()` or `submit_spark_sql_and_wait()` to poll status.

```python theme={null}
from oleander_sdk import SparkSqlSubmitOptions

submitted = await client.submit_spark_sql(SparkSqlSubmitOptions(
    namespace="my-namespace",
    name="nightly-agg",
    query="SELECT day, count(*) AS n FROM oleander.default.events GROUP BY day",
    output_table="default.daily_counts",
))

print(submitted.run_id, submitted.state)  # "..." SUBMITTED
```

### Parameters

<ParamField body="options.namespace" type="str" required>
  Job namespace.
</ParamField>

<ParamField body="options.name" type="str" required>
  Job name.
</ParamField>

<ParamField body="options.query" type="str" required>
  The Spark SQL query to execute.
</ParamField>

<ParamField body="options.output_table" type="str" required>
  The Iceberg table the query result is written to.
</ParamField>

<ParamField body="options.write_mode" type="SparkWriteMode" default="OVERWRITE">
  `OVERWRITE` or `APPEND`. Case-insensitive on input.
</ParamField>

<ParamField body="options.driver_machine_type" type="SparkMachineType" default="spark.1.b">
  Driver machine type.
</ParamField>

<ParamField body="options.executor_machine_type" type="SparkMachineType" default="spark.1.b">
  Executor machine type.
</ParamField>

<ParamField body="options.executor_numbers" type="int" default="2">
  Number of executors, from 1 to 20.
</ParamField>

### Return type: `SparkSqlSubmitResponse`

| Field          | Type              | Description                                                        |
| -------------- | ----------------- | ------------------------------------------------------------------ |
| `run_id`       | `str`             | The ID of the submitted run                                        |
| `state`        | `str`             | Initial run state (for example, `SUBMITTED`)                       |
| `namespace`    | `str`             | Job namespace                                                      |
| `job_name`     | `str`             | Job name                                                           |
| `pipeline_id`  | `Optional[str]`   | Pipeline ID if one exists                                          |
| `submitted_at` | `str`             | ISO timestamp of submission                                        |
| `compute`      | `SparkSqlCompute` | `driver_machine_type`, `executor_machine_type`, `executor_numbers` |
| `output_table` | `str`             | The output table                                                   |
| `write_mode`   | `str`             | The write mode used                                                |

***

## `submit_spark_sql_and_wait(options)`

Submit a Spark SQL query and poll until the run reaches a terminal state (`COMPLETE`, `FAIL`, or `ABORT`). Raises `TimeoutError` if the timeout is exceeded.

```python theme={null}
from oleander_sdk import SubmitSparkSqlAndWaitOptions

result = await client.submit_spark_sql_and_wait(SubmitSparkSqlAndWaitOptions(
    namespace="my-namespace",
    name="nightly-agg",
    query="SELECT day, count(*) AS n FROM oleander.default.events GROUP BY day",
    output_table="default.daily_counts",
    write_mode="APPEND",
))

if result.state != "COMPLETE":
    raise Exception(f"Run {result.run_id} ended with state: {result.state}")
```

Accepts all `submit_spark_sql` options plus:

<ParamField body="options.poll_interval_ms" type="int" default="10000">
  Milliseconds between status polls. Must be greater than 0.
</ParamField>

<ParamField body="options.timeout_ms" type="int" default="600000">
  Maximum time to wait in milliseconds before raising `TimeoutError`.
</ParamField>

### Return type: `SubmitAndWaitResult`

| Field    | Type          | Description                                     |
| -------- | ------------- | ----------------------------------------------- |
| `run_id` | `str`         | The ID of the submitted run                     |
| `state`  | `str`         | Terminal state (`COMPLETE`, `FAIL`, or `ABORT`) |
| `run`    | `RunResponse` | Full run details                                |