Concept
Read a confidence interval as a procedure with assumptions
The interval is useful because of how it was made, not because 95 is a reassuring number.
When it fits
- A report turns a 95% confidence interval into a guarantee about observations or a fixed parameter.
When to avoid it
- Sampling bias or a misspecified model can undermine the procedure despite a precise-looking interval.
The idea
A frequentist confidence level describes the coverage of the interval-making procedure over repeated samples under its assumptions. For an interval around a mean, it does not say that 95% of individual cases lie between the endpoints. Inspect the target, width and method before interpreting it.
An example
An interval for average processing time is not a prediction interval for tomorrow's single job.
Check your result
The interval is interpreted for its stated target without turning it into an individual guarantee.
Keep this limit in mind
- Sampling bias or a misspecified model can undermine the procedure despite a precise-looking interval.
Connected ideas
Useful withKeep the condition attached to a p-value
Evidence and sources
Supports
A frequentist confidence level describes the long-run coverage of the interval-producing procedure under its assumptions.
It is not the proportion of individual observations inside an interval for the population mean.
Confidence Limits for the Mean · Interpretation of confidence limits