Question
Audit the reference class before trusting its base rate
A base rate is only as relevant as the cases allowed into its denominator.
When it fits
- A reference-class forecast produces a compelling percentile or average and the class itself has not been challenged.
When to avoid it
- No reference class is perfectly homogeneous; the goal is a useful outside view with visible limitations, not a claim of exact exchangeability.
A question to ask
What rule put a past case inside or outside the class? · Were failed or cancelled cases captured? · Are duration, cost and completion defined the same way? · Has technology or process changed enough to weaken comparability? · Would another defensible class materially change the forecast?
Why it matters
Inspect the class boundary before using its distribution. Ask which factors genuinely determine comparability, whether selection excluded inconvenient failures, whether the measurement definitions match, and whether structural changes make old cases less relevant. If different reasonable classes give different answers, show that sensitivity.
An example
A 'similar SAP migration' class may become misleading if it mixes greenfield programs with small remediation runs.
Check your result
The forecast includes a defensible class definition and at least one noted limitation or alternate-class sensitivity.
Keep this limit in mind
- No reference class is perfectly homogeneous; the goal is a useful outside view with visible limitations, not a claim of exact exchangeability.
Evidence and sources
Reference class forecasting uses outcomes from comparable past projects as an outside view, but recent review work highlights unresolved issues around reference-class selection and applicability.
A badly chosen class can give a confident but irrelevant base rate.
Reference class forecasting: promises, problems, and a research agenda moving forward · Systematic literature review and limitations