Protocol

Build a matrix without turning timing into proof

An error first seen after a software change is a clue. It does not by itself rule out an older data defect: the change may have exposed it.

A situation to tryIllustrative example

“After” has more than one explanation.

The first failure appears after an update. The update may expose an old invalid value, or create a new bad value.

A working desk with a laptop, external drive, disconnected cable, papers and a coffee mug
Drive and cableThe reports

What would you try?

Two ways to look at it
Exposed old problemA new check rejects an old invalid value.
Created new problemA new mapping creates a bad value.

Try either move. Compare what it can tell you.

Why is the date of the first failure insufficient?

A change can expose a pre-existing defect or introduce one; both can first become visible afterward.

Matrix ratings are not probabilities. Repeated reports are not independent votes, and a completed table does not prove better judgment.

Read the complete idea VedokrokSee a situation. Try a different move.

Before you use it

Use when
  • Several explanations are plausible and discussion keeps counting supporting anecdotes.

Keep this limit in mind

  • Cell ratings are judgments. Document ambiguous evidence and do not convert the matrix into fake precision.

Current editorial reading

Write the prediction behind each rating

Keep evidence as rows and candidate explanations as columns. Before marking a contradiction, write what that explanation predicts under the observed conditions. In a hypothetical migration, compare “new validation rejects an old invalid value” with “the new mapping creates the invalid value.” Check the same input, relevant rule and before/after output. A first failure date alone fits both explanations; it cannot decide between them.

Make the reasoning inspectable

Label weak observations and assumptions, retain an unknown rating, and check whether one disputed row drives the conclusion. Do not count repeated reports of the same event as independent votes or turn consistency marks into probabilities. The primer supplies a comparison method; it does not validate this software example. A 2019 experiment with intelligence analysts found mixed bias-related results, so a tidy matrix is not evidence of better judgment.

Take it with you

A useful matrix shows which prediction an observation tests, what remains compatible and which uncertainty needs a better check. It does not certify the winning explanation.

Sources and evidence (2)

What we checked

Supports

The primer describes comparing evidence across competing hypotheses and checking sensitivity to critical evidence.

Selected passages inspected on 2026-10-05. Method-origin guidance, not a controlled test of this workplace application.

A Tradecraft Primer: Structured Analytic Techniques for Improving Intelligence Analysis (2009) · Analysis of Competing Hypotheses, printed pp. 14–15 (PDF pp. 17–18)

Challenges

Applying ACH reliably removes confirmation bias and improves judgment accuracy.

Selected methods, results and discussion inspected on 2026-10-05. Fifty analysts were randomly assigned; bias-related results were mixed and adherence incomplete. This bounded task does not establish universal ineffectiveness or test the present software example.

The “analysis of competing hypotheses” in intelligence analysis (2019) · Sections 4.1–4.2, 5.1–5.3 and 6

Editorial sources (2)

Editorial version 1.0.0 · Evidence checked 2026-10-05

Save and practise this idea

My learning

Read. Recall. Repeat.

Save this idea and return to it on any device.

Use email
Keep exploring

Connected ideas

You are readingBuild a matrix without turning timing into proof

Original edition evidence and sources

These records belong to the original released edition. The current editorial reading has its own evidence above; neither set silently replaces the other.

Supports

Analysis of Competing Hypotheses compares multiple explanations against evidence and pays particular attention to inconsistent or discriminating evidence.

Matrix judgments remain subjective and depend on the quality and completeness of evidence and hypotheses.

A Tradecraft Primer: Structured Analytic Techniques for Improving Intelligence Analysis · Diagnostic Techniques: Analysis of Competing Hypotheses

All sources (1)
Original edition, downloads and provenance

The reading above is editorial version 1.0.0, bound to this exact source edition. The original data and saved downloads retain that edition and the separate editorial layer. They may contain earlier wording; use the current reading above when interpreting it.

Version 0.1.0

No review details are supplied for this edition.

Read the original edition and overlay as JSON · Release details