Concept

Automation bias

A confident machine answer is still an answer, not independent evidence.

About 1 min read

Keep this limit in mind

  • Blind distrust of automation is also costly. The goal is calibrated reliance, not automatic rejection.

An idea to explore.

Try this idea.

Inspect the example, try the idea in your situation, then check your next move.

Keep reading

The idea

Automation bias is inappropriate reliance on automated advice or cues. It can make us follow a wrong recommendation or miss a problem because the system stayed silent. Automation can improve performance overall; the failure is letting it replace the independent checking the situation still requires.

Steps

  1. Write the decision criteria or a provisional view before seeing automated advice when stakes justify it.
  2. Verify surprising or costly recommendations against raw signals or an independent source.
  3. Know the system's common failure modes instead of relying on a generic reminder to 'be careful'.

An example

An AI risk score says 'low', but one required input is missing. A reviewer who accepts the score because the model produced it is not providing much of a human-in-the-loop.

Check your result

Would you accept the same recommendation if it came from a junior analyst with the same evidence?

Save and practise this idea

My learning

Read. Recall. Repeat.

Save this idea and return to it on any device.

Use email
Keep exploring

More from this collection

Question a cognitive shortcut

Decoy effectDefault effect
Evidence and sources
Provides context

Automation bias is a documented pattern of inappropriate reliance on automated cues or recommendations. It can produce commission errors when a user follows incorrect advice and omission errors when a user fails to act because automation did not signal a problem. This does not mean automation is generally harmful: decision support can improve overall performance, and the relevant question is whether reliance remains calibrated when the system is wrong, incomplete, or difficult to verify.

Archived evidence status: established, context-dependent. Automation bias is a documented pattern of inappropriate reliance on automated cues or recommendations. It can produce commission errors when a user follows incorrect advice and omission errors when a user fails to act because automation did not signal a problem. This does not mean automation is generally harmful: decision support can improve overall performance, and the relevant question is whether reliance remains calibrated when the system is wrong, incomplete, or difficult to verify.

Automation bias: decision making and performance in high-tech cockpits · Automation bias: decision making and performance in high-tech cockpits (1997)

Provides context

Automation bias is a documented pattern of inappropriate reliance on automated cues or recommendations. It can produce commission errors when a user follows incorrect advice and omission errors when a user fails to act because automation did not signal a problem. This does not mean automation is generally harmful: decision support can improve overall performance, and the relevant question is whether reliance remains calibrated when the system is wrong, incomplete, or difficult to verify.

Archived evidence status: established, context-dependent. Automation bias is a documented pattern of inappropriate reliance on automated cues or recommendations. It can produce commission errors when a user follows incorrect advice and omission errors when a user fails to act because automation did not signal a problem. This does not mean automation is generally harmful: decision support can improve overall performance, and the relevant question is whether reliance remains calibrated when the system is wrong, incomplete, or difficult to verify.

Automation bias: a systematic review of frequency, effect mediators, and mitigators · Automation bias: a systematic review of frequency, effect mediators, and mitigators (2011)

Provides context

Automation bias is a documented pattern of inappropriate reliance on automated cues or recommendations. It can produce commission errors when a user follows incorrect advice and omission errors when a user fails to act because automation did not signal a problem. This does not mean automation is generally harmful: decision support can improve overall performance, and the relevant question is whether reliance remains calibrated when the system is wrong, incomplete, or difficult to verify.

Archived evidence status: established, context-dependent. Automation bias is a documented pattern of inappropriate reliance on automated cues or recommendations. It can produce commission errors when a user follows incorrect advice and omission errors when a user fails to act because automation did not signal a problem. This does not mean automation is generally harmful: decision support can improve overall performance, and the relevant question is whether reliance remains calibrated when the system is wrong, incomplete, or difficult to verify.

Automation bias and verification complexity: a systematic review · Automation bias and verification complexity: a systematic review (2017)

Provides context

Automation bias is a documented pattern of inappropriate reliance on automated cues or recommendations. It can produce commission errors when a user follows incorrect advice and omission errors when a user fails to act because automation did not signal a problem. This does not mean automation is generally harmful: decision support can improve overall performance, and the relevant question is whether reliance remains calibrated when the system is wrong, incomplete, or difficult to verify.

Archived evidence status: established, context-dependent. Automation bias is a documented pattern of inappropriate reliance on automated cues or recommendations. It can produce commission errors when a user follows incorrect advice and omission errors when a user fails to act because automation did not signal a problem. This does not mean automation is generally harmful: decision support can improve overall performance, and the relevant question is whether reliance remains calibrated when the system is wrong, incomplete, or difficult to verify.

What is Wrong With Automation Bias? · What is Wrong With Automation Bias? (2026)

All sources (4)
Edition and review

Version 0.1.0 · Published

Edition checked · 2026-11-13

Read this idea as JSON · Release details