Compile an Airflow DAG

Status: Available in ETLantic 0.15.0. ETLantic compiles a plan; it does not install or operate an Airflow scheduler.

Repository examples

Companion scripts under examples/ are not installed with the PyPI wheel. Clone a matching checkout (prefer the v0.15.0 tag) and use uv sync / the documented dependency group before running them.

Install and compile

python -m pip install 'etlantic==0.15.0' 'etlantic-airflow==0.15.0'
git clone https://github.com/eddiethedean/etlantic.git
cd etlantic
python examples/airflow_compile.py

The example first proves local execution, then creates an Airflow profile with a UTC cron schedule and retry policy. It writes examples/_generated_customer_airflow_dag.py.

Complete source: examples/airflow_compile.py.

For CI compilation without running the example:

etlantic validate module.py:Pipeline --profile ./profiles/prod.json --format json
etlantic compile module.py:Pipeline --target airflow --profile ./profiles/prod.json -o dags/

Production profiles require an explicit plugin allowlist in a Profile JSON file (the built-in production name is empty and fail-closed). See Production profiles. Inspect the generated artifact and Airflow import errors before deployment. See Airflow compilation details.