Command-Line Interface¶

Status: Available in ETLantic 0.53.0 (Beta release candidate). This page documents the commands implemented by the installed package.

python -m etlantic --help
python -m etlantic --version

Pipeline targets use package.module:PipelineClass, path/to/file.py:PipelineClass, or a path to an etlantic.pipeline/1 JSON document. Prefer python -m etlantic so the active interpreter is used.

Global options¶

Option	Purpose
`--workspace PATH`	Project/workspace root (default: cwd or `etlantic.toml` parent)
`--ephemeral`	Process-local stores instead of durable `.etlantic/`
`--profile`, `-p`	Default profile for commands that accept `--profile`
`--accept-legacy-bindings`	Allow deprecated profile JSON `bindings` (else `PMCFG111`)
`--verbose` / `-v`, `--quiet` / `-q`	Output verbosity
`--color` / `--no-color`	Colorized output
`--non-interactive`	Do not prompt for confirmation

Profile defaults

When --profile is omitted, the CLI defaults to development (or default_profile from optional etlantic.toml). Pass the same --profile for every command in a workflow.

Durable workspace

By default the CLI writes run reports to .etlantic/reports/ and uses .etlantic/artifacts/ for durable materialization. Pass --ephemeral for process-local stores (0.20 behavior).

Pipeline targets¶

Form	Example	Notes
Module path	`pkg.mod:MyPipeline`	Importable class
File path	`pipeline.py:SamplePipeline`	Import-safe module
Definition JSON	`pipeline.json`	`etlantic.pipeline/1` document

JSON targets load via read_pipeline_json (no code execution during decode). validate and plan accept definition JSON. run of a definition still needs live callables registered in-process (see Programmatic authoring).

python -m etlantic validate pipeline.json --profile development
python -m etlantic plan pipeline.json --profile development --format json

`init`¶

Scaffold a minimal import-safe pipeline project:

python -m etlantic init
python -m etlantic init --directory ./my-pipeline --name SamplePipeline --with-toml

Creates pipeline.py, profiles/<profile>.json, sample JSON under data/, workspace dirs under .etlantic/, and optionally etlantic.toml.

`doctor`¶

Read-only environment, plugin, profile, and workspace checks:

python -m etlantic doctor --profile development
python -m etlantic doctor pipeline.py:SamplePipeline --format json

Exits 0 when checks pass, 16 (ENVIRONMENT_FAILURE) when they fail.

`validate`¶

Validate without executing transformation code:

python -m etlantic validate pipeline.py:SamplePipeline --profile development
python -m etlantic validate pipeline.json --profile development

From a checkout

Repository demos such as examples/memory_customers.py:CustomerPipeline require a git checkout (not on the PyPI wheel). Prefer the init project or a definition JSON file for pip-only workflows.

Options:

--profile, -p: profile name; default development
--format: human, json, or sarif
--allow-adhoc-profile: allow unknown bare profile names (default fails closed with PMCFG100)

Exit 0 when valid, 10 (INVALID_MODEL) on validation errors.

`inspect`¶

Print the logical pipeline graph:

python -m etlantic inspect pipeline.py:SamplePipeline
python -m etlantic inspect pipeline.py:SamplePipeline --format json

`plan`¶

Resolve a deterministic PipelinePlan:

python -m etlantic plan pipeline.py:SamplePipeline --profile development
python -m etlantic plan pipeline.json --profile development --format json

The default output format is JSON. Selection options are:

--run-one NODE
--run-until NODE
--nodes NAME,NAME
--allow-adhoc-profile: allow unknown bare profile names (PMCFG100 otherwise)

--run-one and --run-until are mutually exclusive.

Explain resolution decisions with either form:

python -m etlantic plan explain pipeline.py:SamplePipeline --profile development
python -m etlantic plan pipeline.py:SamplePipeline --profile development --explain

Explain output includes bindings, implementations, capability decisions, and (when selected) portable implementation_kind, ir_fingerprint, and compiler identity for Polars kernel compilation.

`plan optimize`¶

Run advisory optimization passes and emit an explanation / shadow comparison (does not change default plan / run behavior):

python -m etlantic plan optimize pipeline.py:SamplePipeline --profile development
python -m etlantic plan optimize pipeline.py:SamplePipeline --policy shadow
python -m etlantic plan optimize pipeline.py:SamplePipeline --apply-optimizations

--policy is off|shadow|apply_accepted (default: profile policy, else shadow). --apply-optimizations is an alias for --policy apply_accepted and attaches host-consumable annotations only (not a physical rewrite). Production profiles fail closed on empty optimization_pass_allowlist (PMOPT140 → exit 11 / TRUST_FAILURE).

Also: python -m etlantic plan explain … --optimization includes the same optimization artifact in the explain payload.

`plan diff`¶

Compare two resolved plans structurally (targets or plan JSON paths):

python -m etlantic plan diff LEFT RIGHT --profile development --format json

Exit 0 when equal, 15 (BREAKING_CHANGE) when they differ.

`profile`¶

Profile lifecycle helpers:

python -m etlantic profile validate profiles/development.json
python -m etlantic profile show development --format json
python -m etlantic profile diff LEFT.json RIGHT.json
python -m etlantic profile migrate profiles/legacy.json --write

Subcommand	Purpose
`validate`	Schema + semantic checks
`show`	Print resolved profile
`diff`	Compare two profiles
`migrate`	Rewrite legacy `bindings` → `assets`

`run`¶

Validate, plan, and execute with the local runtime:

python -m etlantic run pipeline.py:SamplePipeline --profile development
python -m etlantic run pipeline.py:SamplePipeline --profile development --preview

Supported report formats are text, json, and html. Additional options:

--run-one NODE
--run-until NODE
--intent INTENT
--no-write
--preview: show mutation scope only (no execution)
--allow-adhoc-profile: allow unknown bare profile names (PMCFG100 otherwise)

Reports are written to .etlantic/reports/ unless --ephemeral is set. Keep pipeline modules import-safe (guard side effects under if __name__ == "__main__") so validate / plan do not execute during import. In-memory sources that require seeded data still need a Python companion; file-backed assets (for example json://data/sample.json from etlantic init) run directly via CLI.

`compile`¶

Compile a planned pipeline to an external orchestrator artifact (requires the matching plugin, e.g. etlantic-airflow):

python -m etlantic compile pipeline.py:SamplePipeline \
  --target airflow -o dags/ --profile development
python -m etlantic compile pipeline.py:SamplePipeline \
  --target airflow -o dags/ --preview

From a checkout

The repository also ships examples/memory_customers.py:CustomerPipeline for compile demos; that path is not on the PyPI wheel.

--preview shows mutation scope without writing artifacts.

`generate`¶

Generate ODCS/DTCS/DPCS contract bundles, or emit a pipeline definition JSON:

python -m etlantic generate pipeline.py:SamplePipeline -o contracts/
python -m etlantic generate pipeline.py:SamplePipeline --sqlmodel
python -m etlantic generate pipeline.py:SamplePipeline \
  --kind definition -o pipeline.json

--kind definition writes an etlantic.pipeline/1 document (Available in 0.24). --kind agents writes AGENTS.md, CLAUDE.md, Codex skill, and Cursor rule files (TARGET optional, default .). Existing unmarked files are left alone unless --overwrite is passed; marked user regions are merged. --sqlmodel requires etlantic-sqlmodel. Definition kind works from a class target or an existing definition JSON.

python -m etlantic generate --kind agents
python -m etlantic generate --kind agents path/to/project
python -m etlantic generate --kind agents path/to/project --overwrite

`diff`¶

Diff data contracts, transformations, or pipelines:

python -m etlantic diff PREV CURRENT --kind pipeline --format json
python -m etlantic diff PREV CURRENT --kind data --format sarif

`plugin`¶

python -m etlantic plugin list --profile ./profiles/prod.json --format json
python -m etlantic plugin info polars --kind dataframe
python -m etlantic plugin compatibility etlantic-polars --format json
python -m etlantic plugin compatibility --format human

Supported --kind values today: dataframe, sql, spark, orchestrator, scheduler, transform_compiler, resource, mcp.

plugin compatibility evaluates installed plugin packages (static etlantic-plugin-manifest.json plus packaging metadata) against the core version, protocol ranges, capability vocabulary (etlantic.capabilities/1), plan schema (etlantic.plan/1), Requires-Python, the plugin's etlantic pin, and (when --profile is given) allowlist status. Pass/fail findings use PMPLUG44x codes. Exit code is non-zero when any plugin fails.

Production profiles honor Profile.plugin_allowlist (fail closed). When trust diagnostics include severity error (for example empty allowlist / PMPLUG401), plugin list exits non-zero. plugin info accepts --profile and honors the same allowlist.

`schema`¶

Subcommands: inspect, check, diff, history, impact, acknowledge, propose, monitor. History defaults to .etlantic/schema-history/ and stores fingerprints/metadata only—never source rows.

python -m etlantic schema inspect module:MyContract --format json
python -m etlantic schema check module:MyContract --subject orders --format json
python -m etlantic schema diff PREV CURRENT --format json
python -m etlantic schema history orders --format json
python -m etlantic schema impact PREV CURRENT --format json
python -m etlantic schema propose module:MyContract --subject orders
python -m etlantic schema monitor module:MyContract --subject orders
python -m etlantic schema acknowledge orders --note "accepted additive column"

propose records a candidate observation without mutating contracts. monitor writes an observation into file history. acknowledge accepts a known drift subject for subsequent checks.

`stream`¶

Metadata-only dead-letter and schema-registry operations. Never prints event payloads; unauthorized principals exit 11 (TRUST_FAILURE).

python -m etlantic stream dead-letters inspect --store dlq.json --principal ops
python -m etlantic stream redrive plan --store dlq.json --identity rec-1 --principal ops
python -m etlantic stream schemas check --store registry.json --subject orders-value --fingerprint abc123

inspect and redrive plan read identifier documents. schemas check requires --store (fingerprint identities only) and never registers the candidate it is checking. Store JSON that contains payload keys is rejected (INVALID_MODEL). Production registry adapters require Profile.schema_registry_allowlist.

`context`¶

Assemble a bounded, redacted context bundle for agents. Never executes the pipeline, never resolves secrets, and never contacts the network.

python -m etlantic context bundle pipeline.py:SamplePipeline --format json
python -m etlantic context bundle pipeline.py:SamplePipeline --max-bytes 262144

The payload is etlantic.context_bundle/1. Overflow, missing provenance, stale evidence, or redaction failures emit PMCTX* and a non-zero exit. Tutorial: Human-governed AI.

`proposal`¶

Validate an untrusted etlantic.proposal/1 JSON document in the deterministic sandbox. Does not apply files, submit runs, or create approvals.

python -m etlantic proposal validate proposal.json --format json
python -m etlantic proposal validate proposal.json --target pipeline.py:SamplePipeline

Forbidden actions (run.submit, schedule/DLQ/erasure mutation, secrets) fail closed with PMPROP*. Apply remains etlantic control-plane /v1/approvals*. Tutorial: Human-governed AI.

`schedule`¶

Create and inspect secret-free schedules (interval or 5-field cron). Never embeds payloads or secret values. Tutorial: Scheduler and worker.

python -m etlantic schedule create --store schedules.json --definition-id pipe-1 --interval 60
python -m etlantic schedule list --store schedules.json
python -m etlantic schedule inspect sch-1 --store schedules.json
python -m etlantic schedule pause sch-1 --store schedules.json
python -m etlantic schedule preview sch-1 --store schedules.json
python -m etlantic schedule trigger sch-1 --store schedules.json

`scheduler`¶

Timer-leadership process. Production must not colocate this with the FastAPI gateway.

python -m etlantic scheduler serve --store schedules.json --once

`worker`¶

Execution host. Polls CP3 durable outbox; never imports FastAPI.

python -m etlantic worker serve --durable-store schedules.durable.json --once

Tutorial: Scheduler and worker.

`reliability`¶

Subcommands: freshness, partition-check, repair-explain, backfill-preview, reconcile, plan-diff, env-diff, quality-trends. These are local ops helpers—not a managed reliability product.

python -m etlantic reliability freshness orders --max-age 3600 --observed-age 120
python -m etlantic reliability partition-check orders --keys dt,region \
  --observed 2024-01-01/us,2024-01-01/eu --minimum-count 2
python -m etlantic reliability quality-trends orders --values 0.1,0.2,0.15
python -m etlantic reliability reconcile orders --left 100 --right 100
python -m etlantic reliability env-diff LEFT.json RIGHT.json

reliability plan-diff is deprecated; prefer etlantic plan diff.

`erasure`¶

Governed data-subject erasure helpers (CP4). Plans and status reports use subject-key fingerprints only — never raw subject values.

python -m etlantic erasure plan \
  --subject-key-fingerprint sha256:… \
  --field email --field phone \
  --tenant acme --workspace analytics
python -m etlantic erasure status REQUEST_ID \
  --tenant acme --workspace analytics

Outputs are local ops helpers for erasure planning/status — not a managed erasure product. Subject values must never appear in plans, reports, or audit.

`viz`¶

Subcommands: dot, html, lineage.

python -m etlantic viz dot examples/memory_customers.py:CustomerPipeline -o pipeline.dot
python -m etlantic viz html examples/memory_customers.py:CustomerPipeline -o lineage.html
python -m etlantic viz lineage examples/memory_customers.py:CustomerPipeline --format json

viz lineage is read-only (stdout). viz dot writes a file only when -o / --output is set; otherwise it prints DOT to stdout. viz html always writes a file and defaults to lineage.html when -o is omitted.

`report`¶

Subcommands: list, show, query, export, compare.

python -m etlantic report list
python -m etlantic report show RUN_ID --format text
python -m etlantic report query --pipeline-id my-pipeline --status succeeded --limit 20
python -m etlantic report export RUN_ID --format json --output report.json
python -m etlantic report compare LEFT RIGHT --store .etlantic/reports

`report query`¶

Query durable run reports (or run history when configured) with filters. Read-only — prints matching runs to stdout.

python -m etlantic report query \
  --pipeline-id my-pipeline \
  --status succeeded \
  --since 2024-01-01T00:00:00+00:00 \
  --until 2024-12-31T23:59:59+00:00 \
  --limit 20 \
  --format json

Option	Purpose
`--pipeline-id`	Filter by pipeline id
`--status`	Filter by run status (for example `succeeded`, `failed`)
`--since`	ISO-8601 lower bound on `started_at`
`--until`	ISO-8601 upper bound on `started_at`
`--limit`	Max rows (default `20`)
`--format`	Output format (default `json`)

By default list / show / query / export read the durable store at .etlantic/reports/ (or under --workspace). Separate shell invocations see the same runs. Pass --ephemeral on run (and later report commands) only when you intentionally want process-local storage. report compare --store reads an explicit file-store root.

`watch`¶

Revalidate a workspace on file changes using the no-import static index. Never executes pipelines.

python -m etlantic watch PATH [--interval 1.0] [--once] [--format json|sarif]

Option	Purpose
`PATH`	Workspace root directory to index
`--interval`	Polling interval in seconds (default `1.0`)
`--once`	Index and emit diagnostics once, then exit
`--format`	`json` (default) or `sarif`

Use --once in CI smoke checks. Continuous watch is for local authoring; Ctrl+C stops the loop. Diagnostics share the same PMID* / validation codes as etlantic-lsp. --format sarif maps workspace diagnostics through the shared SARIF renderer.

Exit codes¶

Documented in etlantic.cli.exit_codes:

Code	Name	Typical meaning
`0`	`SUCCESS`	Command succeeded
`1`	`GENERAL_FAILURE`	Unclassified failure
`2`	`USAGE_ERROR`	Bad arguments / usage
`10`	`INVALID_MODEL`	Validation or profile model errors
`11`	`TRUST_FAILURE`	Plugin trust / allowlist failure
`12`	`PLANNING_FAILURE`	Planning failed
`13`	`EXECUTION_FAILURE`	Run failed, timed out, or cancelled
`14`	`PARTIAL_RUN`	Run completed with partial success
`15`	`BREAKING_CHANGE`	Diff / impact / plan-diff breaking
`16`	`ENVIRONMENT_FAILURE`	Doctor / environment / missing tooling

Prefer --format json / SARIF in CI and gate on valid / diagnostic severity in addition to exit codes.

Mutations¶

Command	Mutates workspace?
`validate`, `inspect`, `plan`, `diff`, `plugin`, `doctor`, `watch`, `stream`, `schedule list/inspect/preview`, `scheduler`, `worker`	No (read-only analysis / metadata)
`viz lineage`	No (read-only; prints to stdout)
`viz dot`	Writes only when `-o` / `--output` is set; otherwise stdout
`viz html`	Writes `lineage.html` by default (`-o` overrides the path)
`report list` / `show` / `query` / `compare`	No (read-only)
`init`	Writes scaffold files and `.etlantic/` layout
`generate`	Writes contract files to `-o` / output directory
`compile`	Writes orchestrator artifacts to `-o` (unless `--preview`)
`run`	Executes pipeline side effects; writes `.etlantic/reports/` (unless `--ephemeral` / `--preview`)
`profile migrate --write`	Rewrites profile JSON
`schema monitor` / `acknowledge`	Writes schema history under `.etlantic/schema-history/`
`report export`	Writes the chosen `--output` file

Never pass secret values on the CLI. Profiles and plans must keep secret material as references only.

Command-Line Interface¶

Global options¶

Pipeline targets¶

init¶

doctor¶

validate¶

inspect¶

plan¶

plan optimize¶

plan diff¶

profile¶

run¶

compile¶

generate¶

diff¶

plugin¶

schema¶

stream¶

context¶

proposal¶

schedule¶

scheduler¶

worker¶

reliability¶

erasure¶

viz¶

report¶

report query¶

watch¶