Concept

Congruence bias

A positive result can be nearly useless if every rival explanation predicts it too.

About 1 min read

Keep this limit in mind

  • Positive tests are not automatically bad. They can be informative when competing explanations predict different outcomes.

An idea to explore.

Try this idea.

Inspect the example, try the idea in your situation, then check your next move.

Keep reading

The idea

Congruence bias concerns test choice. We may choose a test that fits the explanation we already favour without asking whether the same result would also fit plausible alternatives. The problem is weak diagnostic value, not positivity itself.

Steps

  1. List at least one plausible alternative before choosing the test.
  2. Write what each explanation predicts for the possible results.
  3. Prefer observations where the explanations make meaningfully different predictions.

An example

You suspect a cache causes a performance problem and run a test that both the cache and a database bottleneck would fail. A failure supports neither explanation very well.

Check your result

If this test is positive, would the main alternative predict the same thing?

Save and practise this idea

My learning

Read. Recall. Repeat.

Save this idea and return to it on any device.

Use email
Keep exploring

More from this collection

Question a cognitive shortcut

Restraint biasLess-is-better effect
Evidence and sources
Provides context

Congruence Bias is a narrower hypothesis-testing pattern than the broad umbrella of confirmation bias. In classic work, people overvalued tests that were likely to return a positive result if their leading hypothesis were true, even when other tests were more diagnostic among competing hypotheses. The useful claim is not that every positive test is irrational: a positive test strategy can be efficient in some environments, and the problem depends on whether the chosen test can actually distinguish the focal hypothesis from alternatives.

Archived evidence status: established in hypothesis-testing tasks; narrower than confirmation bias. Congruence Bias is a narrower hypothesis-testing pattern than the broad umbrella of confirmation bias. In classic work, people overvalued tests that were likely to return a positive result if their leading hypothesis were true, even when other tests were more diagnostic among competing hypotheses. The useful claim is not that every positive test is irrational: a positive test strategy can be efficient in some environments, and the problem depends on whether the chosen test can actually distinguish the focal hypothesis from alternatives.

Diagnosticity of evidence and congruence bias in hypothesis testing · Diagnosticity of evidence and congruence bias in hypothesis testing (1988)

Provides context

Congruence Bias is a narrower hypothesis-testing pattern than the broad umbrella of confirmation bias. In classic work, people overvalued tests that were likely to return a positive result if their leading hypothesis were true, even when other tests were more diagnostic among competing hypotheses. The useful claim is not that every positive test is irrational: a positive test strategy can be efficient in some environments, and the problem depends on whether the chosen test can actually distinguish the focal hypothesis from alternatives.

Archived evidence status: established in hypothesis-testing tasks; narrower than confirmation bias. Congruence Bias is a narrower hypothesis-testing pattern than the broad umbrella of confirmation bias. In classic work, people overvalued tests that were likely to return a positive result if their leading hypothesis were true, even when other tests were more diagnostic among competing hypotheses. The useful claim is not that every positive test is irrational: a positive test strategy can be efficient in some environments, and the problem depends on whether the chosen test can actually distinguish the focal hypothesis from alternatives.

Confirmation, disconfirmation, and information in hypothesis testing · Confirmation, disconfirmation, and information in hypothesis testing (1987)

All sources (2)
Edition and review

Version 0.1.0 · Published

Edition checked · 2026-11-13

Read this idea as JSON · Release details