Anti-pattern
Do not turn a p-value into a truth percentage
Subtracting from one does not turn a test into a belief meter.
Before you use it
- Use when
- Someone translates p = 0.03 as a 97% chance that a claim is true.
Keep this limit in mind
- Properly used statistical tests remain useful. Replacing them with intuition is not the lesson.
An idea to explore.
Try this idea.
Inspect the example, try the idea in your situation, then check your next move.
A question to ask
What are the estimate, uncertainty, design and practical consequence behind this number?
What to look for
A p-value concerns data under a specified statistical model. It is not the probability that the hypothesis is true, and it does not tell you whether an effect is large or useful.
Why it matters
A p-value concerns data under a specified statistical model. It is not the probability that the hypothesis is true, and it does not tell you whether an effect is large or useful.
An example
A tiny improvement in a huge sample may have a small p-value while changing little about the practical choice.
Check your result
Does your conclusion use the result's size and context, not a converted truth score?
Save and practise this idea
My learning
Read. Recall. Repeat.
Save this idea and return to it on any device.
Keep exploring
Connected ideas
You are readingDo not turn a p-value into a truth percentage
Ideas that link here
These ideas link to this one. Check their conditions and limits before use.
Useful with this ideaRead the uncertainty beside the estimate
Evidence and sources
A p-value is not the probability that a hypothesis is true and does not measure effect size or importance.
Claim basis: guidance. Claim limit: This does not make properly used p-values meaningless. Source access: full_text; accessed on: 2026-09-17; inspected sections: ["Three-page statement release; six principles on PDF page 2 visually checked"]. Source note: A methodological statement from 2016, not new empirical research or a ban on significance testing. Agent scope review: ASA principles 2, 3 and 5 reject hypothesis-probability and effect-size interpretations of a p-value. The draft preserves the role of properly used statistical tests.
American Statistical Association Releases Statement on Statistical Significance and P-Values · Six principles, numbers 2, 3 and 5
Edition and review
Version 0.1.0 · Published
Edition checked · 2026-11-12