Serverless Spark in Athena: Run Jobs from Jupyter, VS Code, Airflow
Athena’s new Apache Spark engine provides a fully managed, serverless Spark execution environment that starts in seconds and scales automatically. Data teams can connect via Spark Connect from Jupyter notebooks, VS Code, or dbt‑Airflow pipelines, eliminating cluster provisioning, reducing costs, and speeding up time‑to‑insight.