Template
Name the smallest change worth acting on
Detectable is not the same as worth the migration.
When it fits
- A study or experiment may detect differences too small to justify a real decision.
When to avoid it
- Some safety or rights requirements are not appropriately traded away for an average performance gain.
A template to use
A change would justify [decision] if it improves [outcome] by at least [meaningful amount], without worsening [important constraint].
Why it matters
Define a practically meaningful change before the result arrives. Include implementation costs, consequences and relevant tradeoffs. Then compare the estimated effect and its uncertainty with that threshold, rather than treating statistical significance as an automatic instruction to act.
An example
A tiny speed gain may not justify replacing a reliable workflow that would require weeks of retraining.
Check your result
The decision threshold is explicit and was not chosen solely to flatter the observed result.
Keep this limit in mind
- Some safety or rights requirements are not appropriately traded away for an average performance gain.
Connected ideas
Useful withAsk what a nonsignificant study could have missed
Evidence and sources
Supports
A statistically detectable difference can be too small to matter practically, while a consequential difference can remain uncertain in a small study.
The practical threshold must come from the decision context, not from the p-value.
Quantitative Techniques · Practical Versus Statistical Significance