Cookbook¶
Status: Available in ETLantic 0.21.0. Short recipes for shipped workflows. Prefer these over Design Studies.
First success¶
| Recipe | Link |
|---|---|
| Five-minute local pipeline | Quickstart (etlantic init) |
| Evolve the generated project | First Pipeline |
| Day-two habits | Best practices |
| In-memory SDK demo (checkout) | examples/memory_customers.py |
| JSON and CSV storage | File storage |
Engines¶
| Recipe | Link |
|---|---|
| Pick an engine | Engine selection |
| Polars / Pandas / SQL / PySpark | Tutorials under Execution |
| Polars↔Pandas Gate A interchange | Interchange |
| Portable transform without native impl | Portable transforms |
CI and production¶
| Recipe | Link |
|---|---|
| SARIF validate in CI | CI integration |
| Production profile + allowlist | CI starter JSON / prod.example.json, Production profiles |
| Bounded production checklist | Production readiness |
| Ops pilot | Ops Pilot |
Contracts and plans¶
| Recipe | Link |
|---|---|
| Generate ODCS / DTCS / DPCS | Contract generation |
| Explain a plan | etlantic plan explain TARGET --format json — Planning |
| Diff pipelines or contracts | CLI etlantic diff |
Secrets, reports, diagnostics¶
| Recipe | Link |
|---|---|
| Env / file secrets | Secrets management |
| Run reports | Run reports |
| Common failures | Troubleshooting |
| Diagnostic codes | Diagnostics |
Orchestration¶
| Recipe | Link |
|---|---|
| Compile to Airflow DAGs | Airflow tutorial (compile-only package) |
| Prefect local execute | Prefect |
CLI run vs Python seed¶
Use CLI validate / plan always. For in-memory assets, seed and run in
Python:
runtime = PipelineRuntime()
runtime.memory.seed("customer_source", rows)
CustomerPipeline.run(profile="development", runtime=runtime)
Use CLI run when extracts/loads bind to durable storage (JSON, CSV, SQL).