Protocol
Decompose the judgment before scoring the whole
A single number can hide five different arguments.
When it fits
- A complex case receives one holistic score and reviewers disagree without knowing which part drove the difference.
When to avoid it
- Do not decompose away an emergent property that genuinely depends on the whole pattern; leave room for a documented holistic check.
Why it matters
Break the judgment into a small set of components that jointly matter, score or assess those components before the overall verdict, then integrate them using the agreed process. Decomposition makes inconsistency inspectable and prevents one vivid feature from silently dominating everything else.
Steps
- When two reviewers disagree overall, the team can locate the component where their judgments diverge.
An example
Assess deployment risk separately for reversibility, blast radius, observability and data integrity before assigning the overall risk level.
Check your result
When two reviewers disagree overall, the team can locate the component where their judgments diverge.
Keep this limit in mind
- Do not decompose away an emergent property that genuinely depends on the whole pattern; leave room for a documented holistic check.
Connected ideas
Use beforeMeasure agreement on the criteria, not only the verdict
Evidence and sources
A century of selection research reviewed by Annual Reviews supports decomposing complex judgments into components and applying standards consistently to reduce noise.
Decomposition can omit emergent qualities when components do not capture the real judgment target.
Improving Workplace Judgments by Reducing Noise: Lessons Learned from a Century of Selection Research · Structuring judgment process