Question

Audit the reference class before trusting its base rate

A base rate is only as relevant as the cases allowed into its denominator.

About 1 min read

A small scene. Your next move.

Try this idea.

Make a move, see what changes, then bring it to your own situation.

Keep reading

Before you use it

Use when
  • A reference-class forecast produces a compelling percentile or average and the class itself has not been challenged.

Keep this limit in mind

  • No reference class is perfectly homogeneous; the goal is a useful outside view with visible limitations, not a claim of exact exchangeability.

A question to ask

What rule put a past case inside or outside the class? · Were failed or cancelled cases captured? · Are duration, cost and completion defined the same way? · Has technology or process changed enough to weaken comparability? · Would another defensible class materially change the forecast?

Why it matters

Inspect the class boundary before using its distribution. Ask which factors genuinely determine comparability, whether selection excluded inconvenient failures, whether the measurement definitions match, and whether structural changes make old cases less relevant. If different reasonable classes give different answers, show that sensitivity.

An example

A 'similar SAP migration' class may become misleading if it mixes greenfield programs with small remediation runs.

Check your result

The forecast includes a defensible class definition and at least one noted limitation or alternate-class sensitivity.

Save and practise this idea

My learning

Read. Recall. Repeat.

Save this idea and return to it on any device.

Use email
Keep exploring

Connected ideas

You are readingAudit the reference class before trusting its base rate

Evidence and sources
Supports

Reference class forecasting uses outcomes from comparable past projects as an outside view, but recent review work highlights unresolved issues around reference-class selection and applicability.

A badly chosen class can give a confident but irrelevant base rate.

Reference class forecasting: promises, problems, and a research agenda moving forward · Systematic literature review and limitations

All sources (1)
Edition and review

Version 0.1.0

No review details are supplied for this edition.

Read this idea as JSON · Release details