Skip to content

Migrating from 0.4 to 0.5

Core remains engine-free

Installing etlantic alone does not install Polars, Pandas, PyArrow, or NumPy. Add dataframe backends explicitly:

pip install etlantic-polars
pip install etlantic-pandas

Implementation engines

Local record implementations continue to use "local":

@Normalize.implementation("local")
def normalize_local(rows: list[Row]) -> list[Row]: ...

Dataframe implementations use "polars" or "pandas" and receive native frames:

@Normalize.implementation("polars")
def normalize_polars(rows: pl.DataFrame) -> pl.DataFrame: ...

Profile selection

Profile(name="prod", dataframe_engine="polars")

Missing plugins fail during validation/planning (PMPLAN401 / PMPLAN410), not mid-run.

Registry semantics

The built-in local plugin is a runtime/records path (dataframe=False). It is not a Polars/Pandas substitute.

Reports

Dataframe step metrics (collection, conversion, ownership, row counts) appear under StepRunReport.metadata["dataframe"].