What is column-level lineage?

Column-level lineage is the tracking of the origin and transformations of data at the level of individual columns in a database or data pipeline.

This means understanding where the data in a specific column comes from, what operations have been performed on it (e.g., filtering, aggregation, calculations), and how it relates to the original sources.

Applying column-level lineage is important for:

• Ensuring data quality and reliability.
• Simplifying auditing and compliance with regulatory requirements.
• Assisting in debugging and analyzing the impact of data changes.

For example, if a report has a column "Total Income," column-level lineage will allow tracing which source tables and columns it is derived from, what calculations are applied, and how changes in the source data will affect the final result.