Protocol
Discuss the reasons, then re-rate privately
Discussion is useful for finding what you missed; consensus is not its only valid output.
When it fits
- Independent reviewers disagree and discussion may expose missed evidence but can also create conformity.
When to avoid it
- Discussion can still influence private rerating; the method reduces but does not eliminate social effects.
Why it matters
After independent ratings are collected, compare reasons and evidence rather than arguing immediately for one number. Let reviewers ask questions and inspect overlooked information. Then end discussion and have each person submit a final judgment privately. Compare what changed and which disagreements remain.
Steps
- Reveal the distribution and key reasons after independent scoring.
- Discuss evidence, definitions and overlooked considerations rather than negotiating an average.
- End the discussion before final ratings are entered.
- Collect final ratings privately.
- Distinguish corrected oversights from remaining principled disagreement.
An example
Security reviewers discuss why two people scored a threat differently, then each privately revises or keeps their rating.
Check your result
The process can show which judgments changed because of new information and which disagreements survived informed discussion.
Keep this limit in mind
- Discussion can still influence private rerating; the method reduces but does not eliminate social effects.
Connected ideas
Useful withPreserve disagreement that survives error correction
Evidence and sources
In AMP, participants discussed perspectives and then completed a final review independently so discussion could reveal oversights without requiring consensus.
Private re-rating does not remove prestige, persuasion or shared-information effects that already occurred during discussion.
Analysis from multiple perspectives (AMP): Applying decision hygiene to analysis of musical structure · Discussion and final independent review
The AMP proof of concept found that structured independent analysis, information sharing and private re-evaluation reduced self-identified errors or oversights while preserving meaningful differences in perspective.
The result is context-specific and does not prove the exact procedure improves accuracy in unrelated domains.
Analysis from multiple perspectives (AMP): Applying decision hygiene to analysis of musical structure · Abstract results