Code gets versioned by default now. Nobody emails a Python file back and forth anymore. Datasets are a different story. Most projects still treat the training data as a fixed thing that sits in a folder, unchanged, forever. It rarely stays that way.
What data versioning actually solves
A model’s results depend on two things: the code and the data. Git covers the first. Without data versioning, the second is often untracked, which means a result from three months ago can become impossible to reproduce once the underlying CSV gets edited, appended to, or replaced.
That’s the core problem. A pipeline can pass every test in CI and still produce a different result next week, not because anything in the code changed, but because the data quietly did.
Where this shows up in a real pipeline
The retail ETL pipeline validates raw exports at a data-quality gate before anything reaches the warehouse. That check catches bad data on the way in. It doesn’t, by itself, answer a different question: which exact version of the data produced last Tuesday’s report. Data versioning is the piece that answers that.
Tools like DVC (Data Version Control) or lakeFS extend Git-style versioning to large files and datasets, so a specific data snapshot can be checked out alongside the code that ran against it, the same way you’d check out a commit.
Why this matters more once you retrain
The support ticket triage platform tracks every model iteration in MLflow. That covers the model side. But a retrain also touches the training data, and if the data itself isn’t versioned, a comparison between two model versions can be muddier than it looks. Did the new model improve because of a better architecture, or because the training set quietly grew by ten thousand rows since the last run? Without data versioning, that question is a guess.
What to version, in practice
Full raw datasets are often too large to version like code directly. A few practical patterns:
- Version the transformation logic (the dbt models, the cleaning scripts) in Git, which is usually small and text-based.
- Version a hash or fingerprint of the raw data alongside each model run, so you can at least detect when the underlying data changed, even without storing every historical copy.
- Use a tool built for this (DVC, lakeFS, or a data warehouse’s native time-travel features) when full dataset versioning is genuinely needed, rather than trying to force Git to handle large binary files.
A quick checklist
- If someone asked you to reproduce a result from two months ago, could you get the exact same data back?
- When you retrain a model, do you know whether the training data changed since the last run?
- Is your data-quality validation logic versioned in Git, even if the raw data itself isn’t?
- Would you be able to tell a data-caused performance change apart from a model-caused one?
FAQ
Do I need a tool like DVC for a small portfolio project?
Not necessarily. Even a simple convention, like storing a hash of the dataset alongside each model run’s metadata, gets most of the benefit without adding a new tool.
Is data versioning the same thing as backing up data?
No. A backup preserves one copy. Versioning tracks the history of changes and lets you check out any specific point in that history, the same distinction as a backup versus a Git repository.
Should raw data or processed data be versioned?
Ideally both, but raw data matters more. Processed data can usually be regenerated from raw data plus the transformation code, as long as both of those are tracked.
