Python API Reference¶

Status: Available in ETLantic 0.53.0 (Beta release candidate). Signatures and docstrings are generated from the package source across the pages linked below.

Start here by persona¶

Persona	Start with	Then
Pipeline author	Author essentials below, CLI	Authoring API, Plan and runtime
Plugin author	Protocols	Plugin SDK, Testing plugins
Control-plane embedder	Control plane API	`etlantic-fastapi`
Agent / IDE author	Agents API	Human-governed AI
CI / ops	CLI, Runtime configuration	Ops examples

Recommended imports¶

import etlantic as etl

# Curated authoring surface
pipeline = etl.Pipeline(...)
transform = etl.Transformation.portable(...)

# Lazy namespaces (import on first use)
etl.transform   # portable authoring helpers
etl.authoring   # PipelineDefinition, builders, JSON codecs
etl.service     # AuthoringService / PolicyContext
etl.dataframe   # dataframe plugin protocols
etl.sql         # SQL plugin protocols
etl.testing     # conformance suites and fault injection
etl.quality     # provisional portable quality AST / gates
etl.connectors  # source / sink / storage connector protocols
etl.control_plane  # CP1 identity, authz, memory stores (provisional)
etl.optimization   # advisory optimize_plan / EvidenceStore / explain_optimization
etl.streaming      # bounded expansion, envelopes, record-error policy
etl.agents         # context bundles, proposal sandbox, agent guidance

You may also import curated symbols directly:

from etlantic import (
    Data,
    Input,
    Output,
    Parameter,
    Pipeline,
    PipelineRuntime,
    Load,
    Extract,
    Transformation,
)

DataContractModel was removed from the public root and etlantic.contracts in 0.37.0. Use Data (or ContractModel) instead — see Migration 0.36 → 0.37.

Author essentials (curated root)¶

These symbols are on the curated root facade (from etlantic import … or import etlantic as etl). Prefer owning-module imports for everything else.

Symbol	Module	One-liner
`Data`	`etlantic.contracts`	ContractModel-compatible dataset type
`Input` / `Output` / `Parameter`	`etlantic.ports`	Transformation port markers
`Transformation`	`etlantic.transformation`	Typed transform interface + implementations
`Extract` / `Load`	`etlantic.pipeline`	Pipeline entry / publication boundaries
`Pipeline`	`etlantic.pipeline`	Declarative graph; `validate` / `plan` / `run` / `explain_plan`
`Profile`	`etlantic.profile`	Environment + allowlist + engine selection; `security_mode` for trust
`PipelineRuntime`	`etlantic.lifecycle`	Process-local plugins, memory, reports
`PipelinePlan`	`etlantic.plan`	Immutable secret-free resolved plan (`schema` required on wire)
`plan_pipeline` / `explain_plan`	`etlantic.plan`	Functional planning helpers
`compile_plan`	`etlantic.orchestration`	External orchestrator artifact emission (verifies fingerprint first)
`ValidationReport`	`etlantic.diagnostics`	Structured validate findings
`PipelineRunReport`	`etlantic.reports`	Structured run outcomes
`SecretRef`	`etlantic.secrets`	Runtime-only secret reference

Owning-module helpers (not curated root — import from the module):

Symbol	Import	Notes
`verify_plan_fingerprint` / `deep_freeze`	`from etlantic.plan import …`	Trust-boundary fingerprint check; freeze nested mappings/lists/sets (not full object graphs). See freeze glossary. Not on `import etlantic as etl` curated root.
`resolve_profile`	`from etlantic.profile import resolve_profile`	Removed from root in 0.28
`load_profile` / `write_profile`	`from etlantic.profile import …`	Removed from root in 0.28
Gate A tabular types	`from etlantic.interchange.tabular import …`	Descriptors / fidelity (`etlantic.interchange/1`)
`BackfillRequest`	`from etlantic.reliability_runtime import BackfillRequest`	Demoted if accessed via root

Top-10 API cookbook¶

Parameters, returns, and common failures for the APIs most adopters call. Full signatures: generated pages below. Diagnostic codes: Diagnostics catalog.

API	Key parameters	Returns	Common failures
`Pipeline.validate`	`profile`, `runtime`	`ValidationReport`	Empty production allowlist (`PMPLUG401`); wiring (`PMPIPE`); trust (`PMPLUG`)
`Pipeline.plan`	`profile`, `runtime`	`PipelinePlan`	Missing plugins/assets/capabilities; fingerprint/schema on wire
`Pipeline.run` / `arun`	`profile`, `runtime`, `plan`	`PipelineRunReport`	Fingerprint mismatch; storage/IO policy; cancelled/timeout
`Pipeline.explain_plan`	`profile`	Explain payload	Same planning failures as `plan`
`Pipeline.inspect`	—	Logical graph summary	Model definition errors
`Transformation.step`	port bindings	Symbolic `Step`	Unknown bindings → `ModelDefinitionError`
`Transformation.implementation`	`engine` name	Decorator	Replaces same class/engine in-process
`Profile` / `load_profile`	JSON path or fields	`Profile`	Legacy `bindings` only → `PMCFG111`; invalid `security_mode` — import `load_profile` from `etlantic.profile`
`plan_pipeline`	pipeline + profile	`PipelinePlan`	Same as `Pipeline.plan`
`compile_plan`	plan + target	Artifacts	Missing orchestrator plugin; fingerprint verify

Worked production failure: empty allowlist → PMPLUG401 (CLI exit 11). See Production profiles and Secrets decision tree.

Optional plugins document factories in package READMEs. See Optional Packages.

Generated API pages¶

Authoring — contracts, transformations, pipelines, ports
Plan and runtime — validation, profiles, plan, runtime, storage, secrets, contract interchange, Gate A tabular interchange
Quality — provisional etlantic.quality/1 AST, gates, codecs
Protocols — dataframe, SQL, Spark, orchestration, viz, testing, reliability
Control plane API (CP1) — etl.control_plane, PMCP errors, FastAPI dual surface

Core API Reference scope

MkDocs API generation covers core src/etlantic (and Medallantic where linked). Optional engine packages (etlantic-polars, …) document factories in package READMEs — see Optional Packages.

Core behavioral contracts¶

API	Returns	Important failures / side effects
`Transformation.step(**bindings)`	A symbolic `Step`; no user code runs	Unknown bindings raise `ModelDefinitionError`
`Transformation.portable`	Decorator registering the recommended engine-neutral definition	Authoring errors raise `ModelDefinitionError` (`PMXFORM*`)
`Transformation.implementation(engine)`	Decorator returning an engine-specific escape-hatch callable	Registration replaces same class/engine in-process; not adaptive-execution eligible
`Pipeline.validate(...)`	`ValidationReport`	Does not execute transforms; empty production allowlist fails closed
`Pipeline.plan(...)`	Immutable, secret-free `PipelinePlan`	Missing plugins/assets/capabilities fail planning; nested mappings/lists/sets are frozen via `deep_freeze` (dataclasses/unknown objects unchanged)
`Pipeline.run(...)` / `arun(...)`	`PipelineRunReport`	Verifies plan fingerprint before execution; storage side effects follow the plan
`Pipeline.to_mermaid()`	Mermaid flowchart string	Does not plan or execute

report = CustomerPipeline.validate(profile="development")
report.raise_for_errors()
plan = CustomerPipeline.plan(profile="development")

PipelineRuntime is application-owned. A new process receives an empty memory store unless durable providers are configured.

Stability¶

ETLantic 0.43.0 is a Beta (PyPI) release suitable for documented single-tenant reference deployments (not unrestricted enterprise production). Public compatibility follows the documented 0.x deprecation policy; minor releases may still include announced migrations. Review the changelog and compatibility matrix before upgrading. Narrative CLI docs: CLI.

Compatibility aliases (0.x)¶

ETLantic 0.28 removed the third wave of root facade aliases (sql, profile, lifecycle clusters). Import those symbols from owning modules — see Migration 0.27 → 0.28.

ETLantic 0.27 removed the second wave (reliability, schema_drift, registry). See Migration 0.26 → 0.27.

ETLantic 0.38 removed the remaining demoted root aliases (hard AttributeError). Import from owning modules — see Migration 0.36 → 0.37.

Category	Examples (removed from root in 0.37)	Prefer
Model graph	`Edge`, `Node`, `LogicalGraph`	`etlantic.model`
Policy / I/O	`SafeIoPolicy`, `OutboundPolicy`	`etlantic.io_policy`, `etlantic.outbound`

Removed in 0.28 (raise on root import): col, select, load_profile, write_profile, Inject, FailureAction, and other symbols listed in the migration guide.

Removed in 0.26 (raise on root import): RunRequest, RunSelection, RunIntent, DebugSession, RunStatus, MemoryStorage, diff_pipelines, generate_contracts, and other symbols listed in the migration guide.

See Surface inventory and Removal candidates.