A number on a dashboard looks wrong. Someone asks where it came from. Without data lineage, the honest answer is often “somewhere in the pipeline,” which isn’t an answer at all.
What data lineage actually tracks
Data lineage is the traceable path a piece of data takes from its original source, through every transformation, into its final destination. It answers two directions of the same question: given this number on a dashboard, which raw tables and transformations produced it, and given this raw source table, everything downstream that depends on it.
Why this matters more than it sounds
Without lineage, a schema change or a bad batch of source data can ripple through a pipeline with no way to know what else it touched. A team debugging a wrong number has to manually trace backward through SQL and scripts, which gets slower and more error-prone as a warehouse grows.
Where dbt provides lineage for free
The retail analytics warehouse and Olist analytics engineering project both model their transformations as dbt models, and dbt tracks the dependency graph between models automatically as a byproduct of how ref() works. That dependency graph is a real, usable form of lineage: it shows exactly which upstream tables feed into any given model, without needing a separate lineage tool bolted on afterward.
Why this connects to data-quality tests
When a dbt test fails on one model, lineage is what tells you which downstream models are affected before they silently ship bad numbers. Both warehouse projects run dozens of automated tests, and lineage is what turns a single failed test into an actionable list of exactly what else needs checking, rather than a guess about what might be affected.
Lineage beyond a single tool
dbt’s lineage graph covers transformations within dbt itself. A full end-to-end picture, from a source database through ingestion, through dbt, into a BI tool, usually needs a dedicated lineage tool or metadata layer that can see across those boundaries, since no single tool naturally tracks a table’s journey through systems it doesn’t control.
A quick checklist
- If a specific number looked wrong right now, could you trace it back to its source in minutes, or would it take real digging?
- If a source table’s schema changed, would you know everything downstream that depends on it?
- Are you already getting partial lineage for free from a tool like dbt, without realizing it?
- Does your lineage picture stop at one tool’s boundary, missing steps that happen elsewhere in the pipeline?
FAQ
Do I need a dedicated lineage tool if I’m already using dbt?
Not necessarily for transformations happening entirely within dbt. A dedicated tool becomes more useful once lineage needs to span multiple systems outside dbt’s visibility.
Is data lineage the same as documentation?
Related but different. Documentation describes what a table or field means. Lineage shows the actual technical path data took to get there, which can be generated automatically in a way documentation usually can’t.
Does lineage help with debugging or just compliance?
Both, but debugging is often the more immediate benefit. Tracing a wrong number back to its source quickly is a daily practical use, separate from any regulatory reason to track lineage.
