Question
Audit the reference class before trusting its base rate
A base rate is only as relevant as the cases allowed into its denominator.
A small scene. Your next move.
Try this idea.
Make a move, see what changes, then bring it to your own situation.
Before you use it
- Use when
- A reference-class forecast produces a compelling percentile or average and the class itself has not been challenged.
Keep this limit in mind
- No reference class is perfectly homogeneous; the goal is a useful outside view with visible limitations, not a claim of exact exchangeability.
A question to ask
What rule put a past case inside or outside the class? · Were failed or cancelled cases captured? · Are duration, cost and completion defined the same way? · Has technology or process changed enough to weaken comparability? · Would another defensible class materially change the forecast?
Why it matters
Inspect the class boundary before using its distribution. Ask which factors genuinely determine comparability, whether selection excluded inconvenient failures, whether the measurement definitions match, and whether structural changes make old cases less relevant. If different reasonable classes give different answers, show that sensitivity.
An example
A 'similar SAP migration' class may become misleading if it mixes greenfield programs with small remediation runs.
Check your result
The forecast includes a defensible class definition and at least one noted limitation or alternate-class sensitivity.
Save and practise this idea
My learning
Read. Recall. Repeat.
Save this idea and return to it on any device.
Keep exploring
Connected ideas
You are readingAudit the reference class before trusting its base rate
Ideas that link here
These ideas link to this one. Check their conditions and limits before use.
Use before this ideaBuild an outside-view reference class
Evidence and sources
Reference class forecasting uses outcomes from comparable past projects as an outside view, but recent review work highlights unresolved issues around reference-class selection and applicability.
A badly chosen class can give a confident but irrelevant base rate.
Reference class forecasting: promises, problems, and a research agenda moving forward · Systematic literature review and limitations
Edition and review
Version 0.1.0
No review details are supplied for this edition.