Evaluator Brief¶
A one-page answer for enterprise evaluators and technical decision-makers.
What ETLantic is¶
A typed, contract-driven modeling layer for data pipelines in Python. You define datasets, transformations, and pipelines once; ETLantic validates and plans them; plugins execute.
It is not a dataframe engine, distributed scheduler, warehouse, or secret manager.
What is ready in alpha 0.15.0¶
| Area | Ready? |
|---|---|
Typed authoring (Data, Transformation, Pipeline) |
Yes |
Validation and secret-free PipelinePlan |
Yes |
| ODCS / DTCS / DPCS interchange | Yes |
| Local in-process runtime + run reports | Yes |
| Memory / callable / JSON / CSV / no-write storage | Yes |
| Env + mounted-file secrets | Yes |
| Polars / Pandas plugins | Yes (separate packages) |
SQL plugin (etlantic-sql) |
Yes (PostgreSQL reference) |
PySpark plugin (etlantic-pyspark) |
Yes (local provider; batch production path) |
| Structured Streaming | Experimental |
| Airflow / orchestrator compilation | Yes (etlantic-airflow) |
| DTCS 3.0 portable plan models/profiles | Yes (install dtcs>=0.13,<1; content floor dtcs 0.14.0) |
@Transformation.portable authoring |
Yes (0.11) |
Portable Polars compiler (kernel + relational /1) |
Yes (0.13+) |
Portable PySpark compiler (kernel + relational /1) |
Yes (0.13+) |
Portable Pandas compiler (kernel + relational /1, eager) |
Yes (0.14) |
Portable SQL compiler (kernel + relational /1) |
Yes (0.15) |
| Public portable transform conformance suite | Yes (0.14) |
| Multi-tenant durable orchestration | No |
| Formal SLA / support response times | No |
Security posture¶
- Plans never contain resolved secrets
- SQL plugins use structured compilation with identifier/parameter safety; untrusted raw SQL is out of scope
- Spark session credentials resolve at acquire time and never embed in plans
- Plugin allowlists / version pins are available in 0.9+ via
Profile.plugin_allowlist(production profiles fail closed when empty) - Threat model documents residual Gaps (provenance, artifact/cache isolation, DoS budgets, outbound constraints, unsafe-serialization prohibition)
- Report vulnerabilities privately; alpha has best-effort fixes only
Read Security and the repository security policy. For a bounded pilot topology and required controls, read Production Readiness.
Production readiness gate (do not skip)¶
ETLantic 0.15.0 is alpha. Plugin allowlists being “Available” does not mean the product is production-ready.
Do not productionize until the security release gate closes (targeted for 1.0). Residual Gaps that still block a production claim include:
| Gap (from Security Evaluation) | Why it matters |
|---|---|
| Plugin provenance beyond allowlist/pins | Supply-chain attestation incomplete |
| Artifact and cache isolation | Cross-run / cross-tenant exposure risk |
| Outbound destination constraints | SSRF / exfiltration controls incomplete |
| Denial-of-service budgets | Unbounded planning/load work |
| Unsafe serialization prohibition | Deserialization attack surface |
| In-process multi-tenancy | Explicitly out of scope—use process isolation |
Treat process-local run reports as operational evidence for a single process, not an audit system of record.
How to read status labels in deeper chapters: Documentation Status.
What not to bet on yet¶
- Copying long Airflow design study tutorials into production—use
examples/airflow_compile.pyandetlantic-airflowinstead - Treating Structured Streaming APIs as stable (they are experimental)
- AWS Secrets Manager / Vault (not shipped); OS keyring is available via
etlantic-keyring - Process-local reports as an audit system of record
- Stable 1.0 compatibility guarantees
- Managed Databricks/EMR/Connect Spark providers
- Advanced portable profiles — Polars, PySpark, Pandas, and SQL claim
kernel +
portable-relational/1in 0.13–0.15. Richer profiles graduate one family at a time afterward (0.15 continuation). Keep a native@implementation(...)for profiles outside the advertised claim set, or forportable_transform_policy="native".
Enterprise readiness matrix¶
| Concern | Status in 0.15 |
|---|---|
| License | MIT (core and official plugins) |
| Supported versions / EOL | Best-effort on current alpha line; see SECURITY.md |
| Compliance attestations (SOC2, GDPR cert) | Adopter-owned — not provided |
| Identity / RBAC / SSO | Out of scope — use process and network isolation |
| HA / DR / RPO / RTO | Adopter-owned topology |
| SBOM / signed provenance | Gap — not yet emitted by release automation |
| Audit system of record | Gap — process-local reports are operational evidence only |
| Tested scale | Local/pilot workloads; no published capacity guarantees |
| Upgrade / rollback | Pin exact versions; see Migration 0.13 → 0.14 |
Recommended evaluation path¶
- Installation —
pip install etlantic==0.15.0 - Quickstart or
examples/quickstart.py - Capabilities
- Optional:
examples/portable_polars_kernel.pywithetlantic-polars - Optional:
examples/dataframe_parity.pywith Polars or Pandas - Optional:
examples/sql_to_sql.py(and otherexamples/sql_*.py) withetlantic-sql - Optional:
examples/pyspark_local.pywithetlantic-pyspark - Optional:
examples/airflow_compile.pywithetlantic-airflow - Optional: SparkForge adapter via
uv sync --group sparkforge - Migration 0.13 → 0.14 (and 0.12 → 0.13 when needed)
- Roadmap summary for sequencing
Support channel¶
GitHub issues for bugs and questions. Include ETLantic version, Python version, and a minimal reproduction. Never include credentials or production data. See SUPPORT.md.