“Works on my machine” usually traces back to one thing: an unpinned dependency. A reproducible python environment for machine learning means anyone, on any machine, gets the exact same library versions you had when the result was produced.
Why unpinned dependencies quietly break things
Installing a library without specifying a version grabs whatever is current at install time. Six months later, that same install command can pull a newer version with different default behavior, a changed API, or a subtly different numerical result. None of that shows up as an error. It shows up as a result that’s slightly different from before, for no reason anyone can trace back to code.
The baseline fix: requirements.txt
A pinned requirements.txt, generated with pip freeze, locks every installed package to an exact version. It’s not elegant, but it’s simple, and it’s the minimum a reproducible python environment for machine learning needs. Anyone installing from that file gets the same versions, not just the same package names.
Where requirements.txt falls short
pip freeze captures everything installed, including transitive dependencies pulled in indirectly, which makes the file harder to read and maintain by hand. It also doesn’t resolve dependency conflicts for you. It just records whatever pip happened to install at the time, conflicts and all.
A step up: Poetry or a lockfile-based tool
Poetry (and similar tools like pip-tools) separate the packages you actually asked for from the full resolved dependency tree, tracked in a lockfile. This makes conflicts visible during install rather than surfacing as a runtime error later, and it keeps the human-readable dependency list separate from the exact pinned versions underneath it.
Where this connects to CI
A reproducible environment is what makes CI/CD for data pipelines actually mean something. The network intrusion detection project runs its pipeline through GitHub Actions specifically so results are reproducible on every change, and that reproducibility depends entirely on the CI runner installing the exact same dependency versions every time, not just “the latest of whatever’s listed.”
Containers take this further
Pinned dependencies solve the Python package version problem. They don’t solve everything below that layer, system libraries, the Python version itself, OS-level dependencies. That’s what makes containerizing a machine learning model a complementary, not competing, practice. Pinned dependencies plus a container cover both layers.
A quick checklist
- Does your project have a requirements.txt or lockfile, or does
pip installgrab whatever’s current? - Are the pinned versions actually committed to version control, not just present locally?
- Does your CI pipeline install from the pinned file, rather than a loose list of package names?
- If you’re not using a container, have you documented the Python version and OS the pinned versions were tested against?
FAQ
Is requirements.txt enough, or do I need Poetry?
Requirements.txt is enough for a small project. Poetry earns its complexity once dependency conflicts start appearing or the project grows past a handful of packages.
Should I pin exact versions or minimum versions?
Exact versions for reproducibility of a specific result. Minimum versions are more common for a library meant to be installed alongside other projects with their own constraints.
Does a virtual environment alone make a project reproducible?
No. A virtual environment isolates dependencies from the rest of the system, but without pinned versions inside it, the same isolation problem can still happen over time.
