Concept

Name the noise before choosing the fix

Not all inconsistency comes from the same place.

About 1 min read

Before you use it

Use when
  • A review process is inconsistent and the team proposes one generic remedy.

Keep this limit in mind

  • Real systems often mix all three noise forms, and some variation may represent legitimate perspective differences.

An idea to explore.

Try this idea.

Inspect the example, try the idea in your situation, then check your next move.

Keep reading

The idea

Separate three possibilities. Level noise: reviewers are systematically harsher or more lenient overall. Pattern noise: reviewers react differently to particular case features. Occasion noise: the same reviewer changes with time, context or random variation. Diagnose which pattern dominates before choosing calibration, rubric, aggregation or workflow changes.

An example

Two consultants may agree on average severity but disagree sharply whenever data loss is involved; that is different from one consultant rating everything higher.

Check your result

The proposed process change targets an observed form of variability rather than the word 'noise' in general.

Save and practise this idea

My learning

Read. Recall. Repeat.

Save this idea and return to it on any device.

Use email
Keep exploring

Connected ideas

You are readingName the noise before choosing the fix

Evidence and sources
Supports

Decision-noise research distinguishes level noise, stable pattern noise and occasion noise, which arise from different sources and require different diagnostics.

The taxonomy is a conceptual model; real systems can contain several forms of noise at once.

Human judgment error in the intensive care unit: a perspective on bias and noise · Noise taxonomy

All sources (1)
Edition and review

Version 0.1.0

No review details are supplied for this edition.

Read this idea as JSON · Release details