Concept
Name the noise before choosing the fix
Not all inconsistency comes from the same place.
When it fits
- A review process is inconsistent and the team proposes one generic remedy.
When to avoid it
- Real systems often mix all three noise forms, and some variation may represent legitimate perspective differences.
The idea
Separate three possibilities. Level noise: reviewers are systematically harsher or more lenient overall. Pattern noise: reviewers react differently to particular case features. Occasion noise: the same reviewer changes with time, context or random variation. Diagnose which pattern dominates before choosing calibration, rubric, aggregation or workflow changes.
An example
Two consultants may agree on average severity but disagree sharply whenever data loss is involved; that is different from one consultant rating everything higher.
Check your result
The proposed process change targets an observed form of variability rather than the word 'noise' in general.
Keep this limit in mind
- Real systems often mix all three noise forms, and some variation may represent legitimate perspective differences.
Evidence and sources
Decision-noise research distinguishes level noise, stable pattern noise and occasion noise, which arise from different sources and require different diagnostics.
The taxonomy is a conceptual model; real systems can contain several forms of noise at once.
Human judgment error in the intensive care unit: a perspective on bias and noise · Noise taxonomy