Skip to content

ETLantic

Overview

eddiethedean/etlantic

Examples¶

Green path¶

Installation — pip install etlantic
Quickstart
Runnable scripts below

Pages under Design Studies are aspirational and may show APIs that are not shipped.

Runnable guides (docs)¶

File-backed pipeline — JSON and CSV
Polars
Pandas
SQL
PySpark
Airflow
Airflow Compile — examples/airflow_compile.py
Portable Polars Kernel — examples/portable_polars_kernel.py
SparkForge Adapter — tests/sparkforge/

Runnable scripts (repository `examples/`)¶

These scripts are exercised by CI. From a checkout:

In-memory quickstart¶

uv run python examples/quickstart.py

Portable Polars kernel¶

uv sync --group dataframes
uv run python examples/portable_polars_kernel.py

JSON and CSV storage¶

uv run python examples/file_storage.py

Dataframe parity (Polars / Pandas)¶

uv sync --group dataframes
uv run python examples/dataframe_parity.py polars
uv run python examples/dataframe_parity.py pandas

SQL to SQL¶

uv sync --group sql
uv run python examples/sql_to_sql.py
uv run python examples/sql_boundary_hybrid.py
uv run python examples/sql_transactional_write.py
uv run python examples/sql_failure_recovery.py

Defaults to in-memory SQLite for demos; set ETLANTIC_SQL_URL for PostgreSQL.

Local PySpark¶

uv sync --group pyspark
uv run python examples/pyspark_local.py

Airflow compile¶

uv sync --group airflow
uv run python examples/airflow_compile.py

Design studies (aspirational)¶

The remaining pages in this section explore intended integrations. Each page opens with a Future design warning. They may contain APIs, packages, or commands that go beyond the shipped surface — prefer the runnable guides and examples/ for installable behavior.

Topic	Prefer instead
CSV and JSON through built-in storage	`examples/file_storage.py`
Pandas and Polars pipelines	`examples/dataframe_parity.py`
SQL execution and pushdown	`examples/sql_*.py`
PySpark batch	`examples/pyspark_local.py`
Airflow compilation	Airflow Compile
Graphviz DOT / HTML lineage	`etlantic.viz` / `etlantic viz`
SparkForge migration adapter	SparkForge Adapter
Structured Streaming	Experimental foundation only
Portable PySpark-style transformations	Portable Transformation

Do not use a design study as an installation or API reference. The capabilities page and API reference define the current boundary.