Anti-pattern

Do not turn a p-value into a truth percentage

Subtracting from one does not turn a test into a belief meter.

About 1 min read

Before you use it

Use when
  • Someone translates p = 0.03 as a 97% chance that a claim is true.

Keep this limit in mind

  • Properly used statistical tests remain useful. Replacing them with intuition is not the lesson.

An idea to explore.

Try this idea.

Inspect the example, try the idea in your situation, then check your next move.

Keep reading

A question to ask

What are the estimate, uncertainty, design and practical consequence behind this number?

What to look for

A p-value concerns data under a specified statistical model. It is not the probability that the hypothesis is true, and it does not tell you whether an effect is large or useful.

Why it matters

A p-value concerns data under a specified statistical model. It is not the probability that the hypothesis is true, and it does not tell you whether an effect is large or useful.

An example

A tiny improvement in a huge sample may have a small p-value while changing little about the practical choice.

Check your result

Does your conclusion use the result's size and context, not a converted truth score?

Save and practise this idea

My learning

Read. Recall. Repeat.

Save this idea and return to it on any device.

Use email
Keep exploring

Connected ideas

You are readingDo not turn a p-value into a truth percentage

Evidence and sources
Supports

A p-value is not the probability that a hypothesis is true and does not measure effect size or importance.

Claim basis: guidance. Claim limit: This does not make properly used p-values meaningless. Source access: full_text; accessed on: 2026-09-17; inspected sections: ["Three-page statement release; six principles on PDF page 2 visually checked"]. Source note: A methodological statement from 2016, not new empirical research or a ban on significance testing. Agent scope review: ASA principles 2, 3 and 5 reject hypothesis-probability and effect-size interpretations of a p-value. The draft preserves the role of properly used statistical tests.

American Statistical Association Releases Statement on Statistical Significance and P-Values · Six principles, numbers 2, 3 and 5

All sources (1)
Edition and review

Version 0.1.0 · Published

Edition checked · 2026-11-12

Read this idea as JSON · Release details