Skip to content

End-to-end pilot

Status: Available in ETLantic 0.41.0. Pip-only walkthrough from init through reshape, optional quality, SARIF validate, run, and report query. No repository clone required.

PyPI only

This page continues Quickstart / First Pipeline. Prefer it when you want one path that ends in CI evidence and durable reports.

1. Install and scaffold

python -m venv .venv && source .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install 'etlantic==0.41.0'
mkdir my-pipeline && cd my-pipeline
python -m etlantic init --with-toml

Confirm python -m etlantic --version prints 0.41.0.

2. Reshape transform

Replace the scaffold passthrough with the upper-case reshape from First Pipeline (UpperName / NamedRow). Re-validate and run:

python -m etlantic validate pipeline.py:SamplePipeline \
  --profile development --format json
python -m etlantic run pipeline.py:SamplePipeline --profile development

Expect succeeded and "ADA" / "GRACE" in data/out.json.

3. Optional quality gate

Portable quality (etlantic.quality / make_quality_gate) is available on local / Polars / Pandas paths; SQL and PySpark still fail closed for portable quality compilers. For this pilot you may skip quality and continue, or add a gate after reading API — Quality and What's new in 0.30.

4. Validate as SARIF (CI evidence)

python -m etlantic validate pipeline.py:SamplePipeline \
  --profile development --format sarif > etlantic.sarif

Copy the same command into PR CI. Full workflow: CI integration.

5. Run and query reports

Default run writes under .etlantic/reports/ (durable unless --ephemeral):

python -m etlantic run pipeline.py:SamplePipeline --profile development
python -m etlantic report list
python -m etlantic report query --status succeeded --limit 5 --format json

See Reports and history.

Next