Heuristic
Increase scrutiny when AI feels effortlessly right
Ease of agreement is not evidence of correctness.
When it fits
- A generated answer arrives fluent, familiar and aligned with what you hoped to hear.
When to avoid it
- The cited study reports associations between confidence and self-reported critical thinking; it does not prove that confidence itself causes errors.
Why it matters
Treat high subjective confidence in the AI answer as a reason to run the normal verification step, not as permission to skip it. Ask what evidence the consequential claim depends on, compare it with the source or system state, and look for the kind of failure that fluent wording can hide.
An example
An AI confidently states that a SAP field is replicated by a certain IDoc segment; verify the actual configuration or documentation before changing production logic.
Check your result
The verification burden is determined by stakes and evidence, not by how convincing the prose feels.
Keep this limit in mind
- The cited study reports associations between confidence and self-reported critical thinking; it does not prove that confidence itself causes errors.
Connected ideas
Useful withCross-check the fact before the agent acts on it
Evidence and sources
In the CHI 2025 survey, higher confidence in GenAI was associated with less reported critical thinking, while higher task-specific self-confidence was associated with more reported critical thinking.
These are associations in self-reports, not proof that confidence in AI causes reduced critical thinking.
The Impact of Generative AI on Critical Thinking: Self-Reported Reductions in Cognitive Effort and Confidence Effects From a Survey of Knowledge Workers · Quantitative findings