Before running a single statistical test, you need two competing statements written down clearly. Null hypothesis vs alternative hypothesis is the starting point every hypothesis test builds on, and getting the wording backwards changes what your result actually means.
Null hypothesis vs alternative hypothesis: the core difference
The null hypothesis states that there’s no effect, no difference, nothing interesting happening, it’s the default, boring assumption you’re trying to find evidence against. The alternative hypothesis is the claim you actually suspect is true, that there is an effect or a difference. A test never proves the alternative directly. It only measures whether the evidence is strong enough to reject the null.
How to write a null and alternative hypothesis correctly
Start with the alternative, since that’s usually the actual question motivating the test, then write the null as its direct negation. If the alternative is “model A has higher accuracy than model B,” the null is “model A’s accuracy is not higher than model B’s,” not some unrelated statement. A common mistake is writing a null hypothesis that doesn’t actually oppose the alternative cleanly, which makes the test’s conclusion ambiguous no matter what the data shows.
A real example: comparing two model versions
Suppose the support ticket triage platform‘s retrained model needs to be compared against its predecessor before trusting it with real tickets. The null hypothesis: the new model’s accuracy is no different from the old one’s. The alternative: the new model’s accuracy is genuinely higher. Only after that framing is set does it make sense to run a statistical test and check whether the observed improvement is large enough to reject the null.
Why this connects to random seed variance
A single comparison between two models can look like an improvement purely by chance, the same concern covered in random seed reproducibility. A properly framed null and alternative hypothesis, tested against a spread of results rather than one run, is what separates a real improvement from a lucky roll.
One-tailed vs two-tailed: writing the alternative precisely
A one-tailed alternative claims a difference in a specific direction, “model A is better than model B.” A two-tailed alternative only claims a difference exists, without specifying direction, “model A’s accuracy differs from model B’s.” The choice has to be made before looking at the results, not after, or the test’s stated confidence level stops meaning what it’s supposed to mean.
A quick checklist
- Does your null hypothesis directly oppose your alternative, with no ambiguous middle ground?
- Did you decide one-tailed versus two-tailed before seeing the results, not after?
- Are you testing a claim you can actually measure, rather than something too vague to evaluate statistically?
- Would rejecting the null actually support the specific alternative you wrote, or something broader than the test can justify?
FAQ
Can a hypothesis test prove the null hypothesis true?
No. Failing to reject the null only means there wasn’t enough evidence against it, not that it’s confirmed true.
What’s a common mistake when writing these hypotheses?
Writing an alternative hypothesis that’s too vague to test directly, or a null hypothesis that isn’t the precise logical opposite of the alternative.
Does the null hypothesis always mean “no difference”?
Usually, but not always literally zero, sometimes it’s a specific baseline value the alternative claims differs from, not strictly “nothing happening.”
