Protocol

History helps only when the past is actually comparable

A record of forecast misses can turn estimation from memory into evidence. The catch is that a local base rate is useful only when the old work is similar enough to the new work, and when changes in process have not made the old distribution obsolete.

About 5 min read

Before you use it

Use when
  • Teams repeatedly estimate similar work but every new estimate starts from intuition.

Keep this limit in mind

  • Small or changing samples can mislead; audit comparability before treating history as a stable base rate.

Current editorial reading

Keep the original forecast, not the remembered one

HM Treasury's 2026 Green Book recommends measuring historical forecast errors and using evidence from similar proposals, preferably from the originating organisation when that evidence exists. That principle is easy to weaken in practice: if the old estimate is overwritten after scope changes, the team loses the error it is trying to learn from. Preserve the forecast as it was known at the decision point, the scope and assumptions behind it, and the actual outcome. Then the gap is evidence rather than a story reconstructed after delivery.

The outside view is a correction, not an oracle

Reference class forecasting formalises the same move at a larger scale: compare a new forecast with the actual outcomes of comparable past projects instead of relying only on the plan for this project. Flyvbjerg argued that this outside view can reduce optimism and strategic distortion in major-project forecasts. The transfer to ordinary knowledge work should stay modest. A local baseline is not proof that the next project will repeat the average miss; it is a prior check on whether the current estimate is unusually optimistic or pessimistic compared with relevant history.

Comparability is the hard part

A recent systematic review of reference class forecasting makes the central trade-off explicit. Broad reference classes provide more data but can mix projects that behave differently. Narrow classes improve similarity but leave fewer observations and therefore less stable estimates. The review also reports evidence that reference class forecasting can lose its advantage when similarity between the target project and the reference projects is weak. This is the strongest reason not to turn every past estimate into one company-wide correction factor. Group work by the features that plausibly change delivery behaviour, and keep the uncertainty visible when the sample is small.

Use the baseline to diagnose the system

The most useful local history often shows which component keeps missing. A migration team may estimate build effort well but repeatedly miss approval wait; another team may be accurate until late rework appears. Keep error components such as active effort, waiting, rework, scope change and realised risks when they can be recorded consistently. When tools, workflow, team structure or definition of done changes materially, keep the old data for history but start a visibly new comparison cohort. This is an editorial application of the evidence, not a validated universal forecasting formula.

Three questions before you reuse the baseline

How many past cases are enough? There is no universal cutoff. A few highly comparable cases can be informative, but they do not justify a precise stable base rate; report the spread and keep confidence low.

What if the process changed? Do not delete the older cases, but do not pool them automatically. Mark the change and compare recent work separately until you know whether the old error pattern still applies.

Should I add the average historical miss to every new estimate? Not by default. Historical error is an input to judgment or a formal reference-class method. A single uplift can hide different causes and can be misleading when the reference cases are heterogeneous.

Take it with you

Keep original forecasts and actual outcomes, then compare like with like. Use local error history to challenge the current estimate and find recurring failure components; do not promote a small, mixed or obsolete sample into a universal correction factor.

Sources and evidence (3)

What we checked

Supports

The 2026 Green Book recommends explicit optimism-bias adjustments informed by historical forecast errors from similar proposals, using the originating organisation's evidence where available.

This is UK public-sector appraisal guidance, not a universal estimation standard. The transferable point is to compare forecasts with actual outcomes from relevant historical work rather than rely only on current-plan confidence.

The Green Book (2026) · Optimism bias

Supports

Reference class forecasting uses the actual performance of comparable projects to provide an outside-view correction to project forecasts.

The paper's practical application is large transport infrastructure. It supports the logic of using comparable historical outcomes, not a claim that the same statistical procedure or uplift applies unchanged to small software or knowledge-work tasks.

From Nobel Prize to Project Management: Getting Risks Right · Abstract; reference class forecasting method

Limits

A systematic review of reference class forecasting literature identifies a trade-off between larger but more heterogeneous reference classes and narrower but less statistically reliable classes, and reports that forecast performance depends materially on similarity between the target project and its reference class.

This directly limits the idea that more historical cases automatically create a better baseline. Reference-class construction is itself a judgment problem, especially when projects are rare, changing or poorly classified.

Reference class forecasting: promises, problems, and a research agenda moving forward · Methodology; reference-class construction and similarity discussion

Challenges

The same review finds continuing debate about how much project underperformance is caused by optimism bias versus complexity, uncertainty and forecasting error, and warns against over-promising reference class forecasting as a complete correction.

Historical error data can expose a pattern without identifying its cause. A local baseline should therefore support diagnosis and calibration rather than replace causal analysis of scope, dependencies, rework and process changes.

Reference class forecasting: promises, problems, and a research agenda moving forward · Origins and critiques; 'It is important not to over-promise the advantages offered by RCF'

Editorial sources (3)

Editorial version 1.0.0 · Evidence checked 2026-09-25

Save and practise this idea

My learning

Read. Recall. Repeat.

Save this idea and return to it on any device.

Use email
Keep exploring

Connected ideas

You are readingHistory helps only when the past is actually comparable

Original edition evidence and sources

These records belong to the original released edition. The current editorial reading has its own evidence above; neither set silently replaces the other.

Supports

The 2026 Green Book recommends explicitly accounting for optimism bias in cost, benefit and duration estimates and using historical forecast errors from similar proposals where available.

The guidance is for UK public appraisal; the transferable principle is empirical correction, not a universal percentage uplift.

The Green Book (2026) · Optimism bias

Supports

Supplementary Green Book optimism-bias guidance recommends basing adjustments on data from past or similar projects and collecting local data to improve future estimates.

Reference classes must be genuinely comparable; irrelevant historical projects can make an estimate worse.

Supplementary Green Book Guidance: Optimism Bias · Introduction and making adjustments

All sources (2)
Original edition, downloads and provenance

The reading above is editorial version 1.0.0, bound to this exact source edition. The original data and saved downloads retain that edition and the separate editorial layer. They may contain earlier wording; use the current reading above when interpreting it.

Version 0.1.0

No review details are supplied for this edition.

Read the original edition and overlay as JSON · Release details