Question
Check who could enter the sample
A million answers from the wrong doorway still came through that doorway.
When it fits
- A large survey or dataset is presented as representative because it contains many records.
When to avoid it
- A probability sample also needs correct implementation, response handling and analysis.
A question to ask
Who had a chance to be included? · Which relevant groups were absent from the sampling route? · Does the conclusion stay within the population the design can reasonably describe?
Why it matters
Inspect the route into the sample before admiring its size. Ask which part of the intended population was reachable, eligible and likely to participate. More records can reduce random noise while leaving systematic omissions intact.
An example
Feedback collected only inside a desktop application misses people who could not install or open it.
Check your result
The population claim is justified by the recruitment process, not just the row count.
Keep this limit in mind
- A probability sample also needs correct implementation, response handling and analysis.
Connected ideas
Useful withTreat missing responses as an unanswered questionCompare with
Ask what was randomized
Evidence and sources
Increasing sample size does not by itself correct a sampling process that systematically misses part of the target population.
The appropriate sampling design depends on the population and the question.
Introductory Statistics 2e, 1.2: Data, Sampling, and Variation · Unrepresentative and convenience samples; nonsampling errors