DTCS¶

Overview¶

ETLantic adopts the Data Transformation Contract Standard (DTCS) as the canonical portable representation of transformation contracts.

DTCS defines what a transformation means. ETLantic provides the Python-first authoring experience and generates DTCS artifacts from typed transformation classes.

The DTCS specification is the normative definition of transformation semantics. This document explains how ETLantic integrates with that specification.

Why ETLantic Uses DTCS¶

Transformation logic should be portable.

Developers should define a transformation once and reuse it across different execution engines without changing its logical interface.

DTCS provides a vendor-neutral representation of:

inputs
outputs
parameters
metadata
transformation identity
compatibility information

ETLantic treats DTCS as the portable artifact for transformation definitions.

Architectural Relationship¶

Python Transformation
        │
        ▼
Transformation
        │
        ▼
ETLantic
        │
        ▼
DTCS Artifact
        │
        ▼
Execution Planning

ETLantic owns the Python API.

DTCS owns the portable transformation representation.

Code-First Workflow¶

Developers author transformations in Python.

class NormalizeCustomers(Transformation):
    customers: Input[RawCustomer]
    minimum_age: Parameter[int] = 18
    result: Output[Customer]

ETLantic generates the corresponding DTCS artifact.

In a code-first project, the Python class remains the authoring source of truth.

Contract-First Workflow¶

ETLantic may load existing DTCS artifacts and reconstruct transformation definitions through its public loading APIs.

Whether authored in Python or imported from DTCS, transformations participate in planning and validation the same way.

Generated DTCS Artifacts¶

A transformation may generate:

contracts/
└── transformations/
    └── normalize-customers.dtcs.yaml

Generated artifacts should be deterministic so they can be reviewed in version control and reproduced in CI.

Validation¶

ETLantic validates transformation contracts before execution.

Examples include:

input compatibility
output compatibility
parameter types
implementation signatures
required metadata
DTCS version compatibility

Planning should fail before execution when required DTCS semantics cannot be satisfied.

Portable Transformation Plans (0.11+)¶

DTCS 3.0.0 defines the implementation-independent Transformation Plan (dtcs.transform-plan/2), Rich Portable Analytics profiles, expressions (including bounded lambdas), functions, semantic actions, and engine capability model. DTCS 2.0 identifiers and dtcs.transform-plan/1 remain readable. The dtcs 0.14.0 toolkit publishes the canonical models and conformance surfaces (current normative content floor). ETLantic's install pin remains dtcs>=0.13,<1 and is not itself a content-version claim.

What remains proposed is ETLantic's etlantic.transform/1 ergonomic authoring profile and its backend compiler integrations—not DTCS Transformation Plan semantics themselves.

@Transformation.portable provides the PySpark-inspired Python authoring surface. Generated DTCS may carry the normalized Transformation Plan, while plugins compile it into backend Execution Plans. If ETLantic-specific syntax and DTCS semantics disagree, DTCS remains authoritative.

ETLantic consumes the public DTCS models rather than maintaining a competing plan schema. Compiler readiness still requires ETLantic authoring, capability mapping, plugin protocol, and conformance work in the 0.11-0.15 roadmap.

Identity and Versioning¶

Every published transformation should expose a stable identity.

Typical metadata includes:

identifier
version
description
owner
tags

Compatibility decisions should follow the DTCS specification and ETLantic's planning rules.

Relationship to Implementations¶

DTCS describes the logical transformation.

Execution implementations remain separate.

Portable compiler implementations also remain separate: the DTCS artifact describes semantic intent, while Polars, Pandas, SQL, and PySpark plugins provide backend lowering.

@NormalizeCustomers.implementation("polars")
def normalize(...):
    ...

@NormalizeCustomers.implementation("pandas")
def normalize(...):
    ...

Multiple implementations may satisfy the same DTCS contract.

Relationship to ODCS and DPCS¶

The three standards complement one another.

ODCS
  Data Contracts
      │
      ▼
DTCS
  Transformation Contracts
      │
      ▼
DPCS
  Pipeline Contracts

ETLantic unifies these standards through strongly typed Python models.

Design Principles¶

ETLantic follows these principles when integrating with DTCS:

Python classes are the preferred authoring interface.
DTCS is the portable artifact.
Types define transformation interfaces.
Implementations are interchangeable.
Planning precedes execution.
Generated artifacts are deterministic.