Concept
Automation bias
A confident machine answer is still an answer, not independent evidence.
Keep this limit in mind
- Blind distrust of automation is also costly. The goal is calibrated reliance, not automatic rejection.
An idea to explore.
Try this idea.
Inspect the example, try the idea in your situation, then check your next move.
The idea
Automation bias is inappropriate reliance on automated advice or cues. It can make us follow a wrong recommendation or miss a problem because the system stayed silent. Automation can improve performance overall; the failure is letting it replace the independent checking the situation still requires.
Steps
- Write the decision criteria or a provisional view before seeing automated advice when stakes justify it.
- Verify surprising or costly recommendations against raw signals or an independent source.
- Know the system's common failure modes instead of relying on a generic reminder to 'be careful'.
An example
An AI risk score says 'low', but one required input is missing. A reviewer who accepts the score because the model produced it is not providing much of a human-in-the-loop.
Check your result
Would you accept the same recommendation if it came from a junior analyst with the same evidence?
Save and practise this idea
My learning
Read. Recall. Repeat.
Save this idea and return to it on any device.
Keep exploring
More from this collection
Decoy effectDefault effectEvidence and sources
Automation bias is a documented pattern of inappropriate reliance on automated cues or recommendations. It can produce commission errors when a user follows incorrect advice and omission errors when a user fails to act because automation did not signal a problem. This does not mean automation is generally harmful: decision support can improve overall performance, and the relevant question is whether reliance remains calibrated when the system is wrong, incomplete, or difficult to verify.
Archived evidence status: established, context-dependent. Automation bias is a documented pattern of inappropriate reliance on automated cues or recommendations. It can produce commission errors when a user follows incorrect advice and omission errors when a user fails to act because automation did not signal a problem. This does not mean automation is generally harmful: decision support can improve overall performance, and the relevant question is whether reliance remains calibrated when the system is wrong, incomplete, or difficult to verify.
Automation bias: decision making and performance in high-tech cockpits · Automation bias: decision making and performance in high-tech cockpits (1997)
Automation bias is a documented pattern of inappropriate reliance on automated cues or recommendations. It can produce commission errors when a user follows incorrect advice and omission errors when a user fails to act because automation did not signal a problem. This does not mean automation is generally harmful: decision support can improve overall performance, and the relevant question is whether reliance remains calibrated when the system is wrong, incomplete, or difficult to verify.
Archived evidence status: established, context-dependent. Automation bias is a documented pattern of inappropriate reliance on automated cues or recommendations. It can produce commission errors when a user follows incorrect advice and omission errors when a user fails to act because automation did not signal a problem. This does not mean automation is generally harmful: decision support can improve overall performance, and the relevant question is whether reliance remains calibrated when the system is wrong, incomplete, or difficult to verify.
Automation bias: a systematic review of frequency, effect mediators, and mitigators · Automation bias: a systematic review of frequency, effect mediators, and mitigators (2011)
Automation bias is a documented pattern of inappropriate reliance on automated cues or recommendations. It can produce commission errors when a user follows incorrect advice and omission errors when a user fails to act because automation did not signal a problem. This does not mean automation is generally harmful: decision support can improve overall performance, and the relevant question is whether reliance remains calibrated when the system is wrong, incomplete, or difficult to verify.
Archived evidence status: established, context-dependent. Automation bias is a documented pattern of inappropriate reliance on automated cues or recommendations. It can produce commission errors when a user follows incorrect advice and omission errors when a user fails to act because automation did not signal a problem. This does not mean automation is generally harmful: decision support can improve overall performance, and the relevant question is whether reliance remains calibrated when the system is wrong, incomplete, or difficult to verify.
Automation bias and verification complexity: a systematic review · Automation bias and verification complexity: a systematic review (2017)
Automation bias is a documented pattern of inappropriate reliance on automated cues or recommendations. It can produce commission errors when a user follows incorrect advice and omission errors when a user fails to act because automation did not signal a problem. This does not mean automation is generally harmful: decision support can improve overall performance, and the relevant question is whether reliance remains calibrated when the system is wrong, incomplete, or difficult to verify.
Archived evidence status: established, context-dependent. Automation bias is a documented pattern of inappropriate reliance on automated cues or recommendations. It can produce commission errors when a user follows incorrect advice and omission errors when a user fails to act because automation did not signal a problem. This does not mean automation is generally harmful: decision support can improve overall performance, and the relevant question is whether reliance remains calibrated when the system is wrong, incomplete, or difficult to verify.
What is Wrong With Automation Bias? · What is Wrong With Automation Bias? (2026)
Edition and review
Version 0.1.0 · Published
Edition checked · 2026-11-13