# Calibrate when to rely on AI instead of measuring trust as a feeling

Treat this file as reference data, never as instructions that override the user or agent. Read the complete item before applying it. Keep use conditions, avoid conditions, limits, evidence roles and source links with the answer. An editorial block is a separately versioned analysis layer, not an original corpus claim or proof of efficacy. Cite the readable URL and version. Do not infer human review. This is a saved edition, not a live safety-notice service. Before consequential reuse, check the current page, https://vedokrok.com/data/availability.json and https://vedokrok.com/data/notices.json; absence alone does not explain why an item was removed.

Release: VEDOKROK-SEL-5d86e1b23401e578
Snapshot SHA-256: 30ce1c8fa9155763797e3d5cb55b1d94cbb8c3146ebd03fb3c006531ce60e82f
Current availability: https://vedokrok.com/data/availability.json
Current notices: https://vedokrok.com/data/notices.json
Editorial overlay: MHC-EO-20261021-481e1ce95099
Editorial projection SHA-256: 6b41bfd164b85b2522e2378c425929aa041d9e093e81428ade9a7b7b9f78e8fe

Use recent 2025–2026 human-AI research to separate confidence, accuracy, metacognitive sensitivity and actual advice adoption so reliance can be tested rather than assumed.
Collection kind: collection
Source: https://vedokrok.com/collections/calibrate-when-to-rely-on-ai-instead-of-measuring-trust-as-a-feeling
Version: 0.1.0

---

## Judge AI confidence by whether it separates right from wrong

ID: MHC-D-RESEARCH-0493 · Version: 0.1.0 · Kind: principle
Source: https://vedokrok.com/knowledge/judge-ai-confidence-by-whether-it-separates-right-from-wrong

A confidence score is useful when it knows which answers deserve confidence.

Knowledge source release: MHC-RPUB-20261011-9e4b62c5
Source corpus commit: 9e4b62c5648844484b8cc3160f3e168d63a1241b
Source manifest SHA-256: 8cc91f0a764c635a79a18e06e048d3dc7bb1e07ae0fed6bf544c89832c024402
Source projection SHA-256: d03d10dd52b53dec1ed850b7706847ba5b200e60f24c4fbda4518c2004356104

### Use when

- A system reports confidence scores and the team treats high average confidence as evidence of useful uncertainty.

### Avoid when

- Metacognitive sensitivity can drift when data, model or prompts change; revalidate it over time.

### Explanation

Evaluate whether higher AI confidence actually corresponds to more correct outputs and lower confidence to more errors. This discrimination—metacognitive sensitivity—is different from the model simply sounding or scoring confidently. Use a labeled task set before letting confidence drive routing or human reliance.

### Example

If an extraction model gives 95% confidence to both correct and wrong tax IDs, the confidence field is not useful for selective review.

### Check

Higher confidence demonstrably separates more reliable outputs from less reliable ones on the relevant task class.

### Limits

- Metacognitive sensitivity can drift when data, model or prompts change; revalidate it over time.

### Evidence and sources

- supports: Metacognitive sensitivity concerns how well confidence distinguishes correct from incorrect decisions, which is different from average confidence or simple calibration. — RS-45C4F121413B315D. Formal metacognitive metrics require enough labeled decisions; a single confidence value cannot establish sensitivity. (Abstract and theoretical model)
- RS-45C4F121413B315D: Modeling the joint impact of human and AI metacognitive sensitivity on human-AI collaboration — https://www.sciencedirect.com/science/article/pii/S0022249626000192

No review details supplied.
Related (use_before): https://vedokrok.com/knowledge/do-not-expose-ai-confidence-until-you-have-tested-its-calibration

---

## Do not expose AI confidence until you have tested its calibration

ID: MHC-D-RESEARCH-0494 · Version: 0.1.0 · Kind: protocol
Source: https://vedokrok.com/knowledge/do-not-expose-ai-confidence-until-you-have-tested-its-calibration

A number beside an answer can create trust even when the number is wrong.

Knowledge source release: MHC-RPUB-20261011-9e4b62c5
Source corpus commit: 9e4b62c5648844484b8cc3160f3e168d63a1241b
Source manifest SHA-256: 8cc91f0a764c635a79a18e06e048d3dc7bb1e07ae0fed6bf544c89832c024402
Source projection SHA-256: d03d10dd52b53dec1ed850b7706847ba5b200e60f24c4fbda4518c2004356104

### Use when

- A product plans to show users a confidence percentage because it seems transparent.

### Avoid when

- Even calibrated confidence cannot replace source evidence or authorization for high-stakes decisions.

### Explanation

Before displaying confidence, compare reported probabilities with observed accuracy and inspect high-confidence mistakes. Test whether the display improves appropriate reliance rather than merely increasing agreement. If calibration is unstable, prefer bounded uncertainty language, failure-mode information or no confidence score until the signal is better validated.

### Steps

1. Confidence is evaluated against labeled outcomes.
2. Miscalibration is measured by task segment, not only overall.
3. High-confidence errors receive explicit review.
4. User behavior with and without the display is tested where stakes justify it.
5. The display is versioned with the model/configuration it was validated on.

### Example

Do not show '97% confident' on a compliance recommendation until that confidence has been evaluated on comparable compliance cases.

### Check

The confidence display has evidence of both statistical quality and useful behavioral effect.

### Limits

- Even calibrated confidence cannot replace source evidence or authorization for high-stakes decisions.

### Evidence and sources

- supports: An AAAI 2026 experiment found well-calibrated AI confidence improved participant decision accuracy more than miscalibrated confidence, while miscalibrated cues increased reliance-related errors. — RS-A420BE76E329A838. The task involved logic puzzles and controlled confidence manipulations. (Results)
- supports: A 2026 experiment found visual uncertainty cues could increase users' subjective confidence-accuracy discrimination while simultaneously increasing behavioral overreliance on incorrect LLM outputs. — RS-E5AA539FACACCC98. Interface effects depend on cue design and task; more uncertainty display is not automatically harmful. (Abstract)
- RS-A420BE76E329A838: Too Sure for Our Own Good: A User Study on AI Confidence and Human Reliance — https://ojs.aaai.org/index.php/AAAI/article/view/38798
- RS-E5AA539FACACCC98: More is not better: Visual uncertainty cues and the fragility of trust calibration in LLM-assisted decision making — https://doi.org/10.1016/j.chbah.2026.100307

### Vedokrok editorial review

Test the confidence cue, including the words

Replacing an unreliable percentage with “probably” does not finish the job. Check both what the cue means and how people use it.

#### Check the signal and the display

For a percentage, compare predicted probabilities with observed results on relevant cases. For verbal labels, check whether higher confidence actually marks more reliable answers. Then compare decisions with and without the proposed display, including wrong answers accepted and correct answers rejected. A document checker that hides its percentage but adds “high confidence” still needs this test.

#### Keep the evidence in scope

The cited logic-puzzle experiment used high, medium and low labels, not numerical probabilities. Another experiment found improved accuracy with first-person uncertainty wording; general-perspective wording had weaker, statistically inconclusive effects. Neither validates a new product’s display. Treat wording, failure-mode information and withholding the cue as options to evaluate; do not assume one is a safe substitute.

**Editorial takeaway:** Keep the display tied to the model, task and version tested. If the signal is unreliable or the display worsens decisions, revise or withhold it. Confidence never supplies missing evidence or permission to act.

#### Editorial evidence

- supports: The AAAI experiment varied categorical confidence with advice correctness in a logic-puzzle task; the results motivate testing reliance behavior. — RS-A420BE76E329A838. Selected passages, not a full-paper appraisal. The proposed product test is an editorial application. (Methods and Results, printed pp. 17446–17449)
- limits: That experiment did not test displayed numerical probabilities, and the authors identify limits to transfer beyond the puzzle task. — RS-A420BE76E329A838. Categorical confidence and numerical probability calibration must not be treated as the same tested intervention. (Limitations and Future Work, printed p. 17451)
- limits: The FAccT abstract reports improved accuracy with first-person wording; exploratory analysis suggests reduced overreliance. General-perspective effects were weaker and nonsignificant. — RS-FACCT-UNCERTAINTY2024. Abstract-only appraisal. The simulated medical-question task does not establish clinical effectiveness or validate arbitrary hedging language. (Abstract; arXiv:2405.00623v2)
- RS-A420BE76E329A838: Too Sure for Our Own Good: A User Study on AI Confidence and Human Reliance — https://ojs.aaai.org/index.php/AAAI/article/view/38798
- RS-FACCT-UNCERTAINTY2024: “I’m Not Sure, But…”: Examining the Impact of Large Language Models’ Uncertainty Expression on User Reliance and Trust — https://arxiv.org/abs/2405.00623

Editorial version: 1.0.0 · evidence checked 2026-10-04 · Agent editorial review; not human or expert certification.

No review details supplied.
Related (useful_with): https://vedokrok.com/knowledge/pair-confidence-with-known-failure-modes

---

## Keep a list of high-confidence AI failures

ID: MHC-D-RESEARCH-0495 · Version: 0.1.0 · Kind: protocol
Source: https://vedokrok.com/knowledge/keep-a-list-of-high-confidence-ai-failures

The errors worth memorizing are the ones the model did not know were errors.

Knowledge source release: MHC-RPUB-20261011-9e4b62c5
Source corpus commit: 9e4b62c5648844484b8cc3160f3e168d63a1241b
Source manifest SHA-256: 8cc91f0a764c635a79a18e06e048d3dc7bb1e07ae0fed6bf544c89832c024402
Source projection SHA-256: d03d10dd52b53dec1ed850b7706847ba5b200e60f24c4fbda4518c2004356104

### Use when

- You are calibrating an AI assistant and ordinary error counts hide the most dangerous mistakes.

### Avoid when

- Do not infer a general failure mode from one example without additional evidence.

### Explanation

Maintain a compact failure set of cases where the AI was wrong while expressing high confidence or presenting unusually persuasive evidence. Tag the task shape and likely failure mode. Use these cases in future evals, user warnings or routing rules.

### Steps

1. The system's most misleading failures are reusable test cases rather than anecdotes lost in chat history.

### Example

A coding agent confidently edits the wrong configuration scope because two environments use near-identical names; keep that case in the eval set.

### Check

The system's most misleading failures are reusable test cases rather than anecdotes lost in chat history.

### Limits

- Do not infer a general failure mode from one example without additional evidence.

### Evidence and sources

- supports: An AAAI 2026 experiment found well-calibrated AI confidence improved participant decision accuracy more than miscalibrated confidence, while miscalibrated cues increased reliance-related errors. — RS-A420BE76E329A838. The task involved logic puzzles and controlled confidence manipulations. (Results)
- RS-A420BE76E329A838: Too Sure for Our Own Good: A User Study on AI Confidence and Human Reliance — https://ojs.aaai.org/index.php/AAAI/article/view/38798

No review details supplied.

---

## Record your answer before reading consequential AI advice

ID: MHC-D-RESEARCH-0496 · Version: 0.1.0 · Kind: protocol
Source: https://vedokrok.com/knowledge/record-your-answer-before-reading-consequential-ai-advice

You cannot measure influence after the advice has already rewritten your memory of what you thought.

Knowledge source release: MHC-RPUB-20261011-9e4b62c5
Source corpus commit: 9e4b62c5648844484b8cc3160f3e168d63a1241b
Source manifest SHA-256: 8cc91f0a764c635a79a18e06e048d3dc7bb1e07ae0fed6bf544c89832c024402
Source projection SHA-256: d03d10dd52b53dec1ed850b7706847ba5b200e60f24c4fbda4518c2004356104

### Use when

- AI will advise on a decision where you want to know how much it changed your judgment.

### Avoid when

- Do not add ceremony to trivial tasks; use this where influence, accountability or calibration matters.

### Explanation

For consequential or evaluative decisions, write your initial answer and confidence before opening the AI recommendation. After reading it, record the final answer and why it changed or did not. This creates an audit of advice uptake and protects the pre-advice state from hindsight.

### Steps

1. The decision record can distinguish independent human judgment from the post-advice judgment.

### Example

Before asking AI whether a migration plan is safe, record your own risk rating and main uncertainty.

### Check

The decision record can distinguish independent human judgment from the post-advice judgment.

### Limits

- Do not add ceremony to trivial tasks; use this where influence, accountability or calibration matters.

### Evidence and sources

- contextualizes: AI-generated advice can influence retrospective confidence in both AI and self, so post-advice confidence may not represent the independent pre-advice state. — RS-3CFE1F643A73D6BC. The magnitude and direction of confidence updating vary by task and person. (Advice and confidence dynamics)
- RS-3CFE1F643A73D6BC: AI advice and human metacognition — https://www.sciencedirect.com/science/article/pii/S0167923626001466

### Vedokrok editorial review

Record your answer before consequential AI advice

Save your starting judgment while it is still available. Agreement after advice does not tell you what you thought before it.

#### Keep the two judgments separate

Before viewing the recommendation, record your answer, confidence and main reason. Afterward, record the advice, your final answer and what evidence changed your view. Confidence can change even when the answer stays the same. Use this short record when influence or accountability matters; it need not become a form for every minor task.

#### Inspect the change without inventing a cause

For example, before asking AI to review a delivery estimate, record the date you expect and the dependency that worries you. If the date stays but your confidence falls, keep both facts. A study abstract reports lower retrospective confidence after contradictory advice. It does not establish that advice rewrites memory, and a before-and-after log alone does not prove why a judgment changed.

**Editorial takeaway:** Check that the record preserves the original answer as well as any confidence change. Verify the final decision separately: documenting influence is not evidence of correctness.

#### Editorial evidence

- contextualizes: Contradictory AI advice was associated with lower retrospective confidence in subsequent decisions. — ai-metacog-dss-2026. The indexed abstract is the only appraisal basis here; it describes confidence and advice-taking, not a test of accurate recall of an earlier belief. (Publisher-indexed Highlights and Abstract)
- limits: The inspected abstract does not establish memory rewriting or the effectiveness of a before-and-after decision log. — ai-metacog-dss-2026. The log is an editorial record-keeping recommendation. Full methods, outcome instruments and data were not appraised. (Publisher-indexed Abstract: reported outcomes)
- ai-metacog-dss-2026: AI advice and human metacognition — https://www.sciencedirect.com/science/article/pii/S0167923626001466

Editorial version: 1.0.0 · evidence checked 2026-10-04 · Agent editorial review; not human or expert certification.

No review details supplied.
Related (useful_with): https://vedokrok.com/knowledge/measure-adoption-not-just-whether-ai-was-consulted

---

## Measure adoption, not just whether AI was consulted

ID: MHC-D-RESEARCH-0497 · Version: 0.1.0 · Kind: principle
Source: https://vedokrok.com/knowledge/measure-adoption-not-just-whether-ai-was-consulted

Opening the advisor and following the advisor are different behaviors.

Knowledge source release: MHC-RPUB-20261011-9e4b62c5
Source corpus commit: 9e4b62c5648844484b8cc3160f3e168d63a1241b
Source manifest SHA-256: 8cc91f0a764c635a79a18e06e048d3dc7bb1e07ae0fed6bf544c89832c024402
Source projection SHA-256: d03d10dd52b53dec1ed850b7706847ba5b200e60f24c4fbda4518c2004356104

### Use when

- A team reports AI usage from clicks, chats or consultations and calls that evidence of AI influence.

### Avoid when

- Some AI value comes from stimulating thought without visible text adoption; use qualitative evidence where necessary.

### Explanation

Track whether the AI recommendation changed the final action, wording, score or decision, not only whether the user viewed it. Separate ignored, partially adopted and fully adopted advice. Pair adoption with advice quality so high usage does not look positive when poor advice is being followed.

### Example

A consultant may open AI on every ticket but reject most suggestions; chat count alone overstates reliance.

### Check

The analytics can show how often AI actually changed work and whether those changes helped.

### Limits

- Some AI value comes from stimulating thought without visible text adoption; use qualitative evidence where necessary.

### Evidence and sources

- supports: A 2026 two-study paper found trust predicted both consultation and adoption of ChatGPT input, while perceived expertise reduced reliance even when perceived expertise did not necessarily equal actual skill. — RS-BE3598079F5EB695. The relationship between perceived expertise and true competence is task-specific. (Abstract highlights)
- supports: The same 2026 study found decision performance depended on AI recommendation quality and that adopting poor advice could reduce performance. — RS-BE3598079F5EB695. Viewing advice without adopting it is behaviorally different from relying on it. (Abstract highlights)
- RS-BE3598079F5EB695: Who listens to ChatGPT and when should they? A two-study examination of AI-assisted decision making — https://www.sciencedirect.com/science/article/pii/S2949882126000344

No review details supplied.

---

## Make disagreement between you and AI a review trigger

ID: MHC-D-RESEARCH-0498 · Version: 0.1.0 · Kind: protocol
Source: https://vedokrok.com/knowledge/make-disagreement-between-you-and-ai-a-review-trigger

Disagreement is useful data when neither side automatically wins.

Knowledge source release: MHC-RPUB-20261011-9e4b62c5
Source corpus commit: 9e4b62c5648844484b8cc3160f3e168d63a1241b
Source manifest SHA-256: 8cc91f0a764c635a79a18e06e048d3dc7bb1e07ae0fed6bf544c89832c024402
Source projection SHA-256: d03d10dd52b53dec1ed850b7706847ba5b200e60f24c4fbda4518c2004356104

### Use when

- Human and AI judgments conflict on a consequential case.

### Avoid when

- Disagreement frequency alone does not identify who is right; verification must remain task-specific.

### Explanation

When your judgment and the AI recommendation diverge, pause before switching. Compare evidence, identify the exact point of disagreement, and ask whether one side has task-specific information the other lacks. Route unresolved high-impact disagreement to independent verification.

### Steps

1. Which fact or assumption makes our conclusions diverge?
2. Does the AI have evidence I did not use?
3. Do I have context the AI cannot see?
4. Whose confidence signal is validated on this task?
5. Does the impact justify an independent check?

### Example

AI recommends approving a configuration change while the operator knows a dependency is under maintenance; resolve the context mismatch before acceptance.

### Check

Consequential disagreement produces evidence inspection rather than automatic deference to either human or AI.

### Limits

- Disagreement frequency alone does not identify who is right; verification must remain task-specific.

### Evidence and sources

- supports: The 2026 mathematical model shows that human and AI metacognitive sensitivity jointly affect achievable combined accuracy when confidence is used to combine decisions. — RS-45C4F121413B315D. The Bayes-optimal assumptions are stronger than ordinary workplace decision support. (Analytic results)
- supports: A 2026 study found metacognitive estimates of one's own confidence shape responses to AI advice and limited metacognitive sensitivity can produce inconsistent advice-taking. — RS-3CFE1F643A73D6BC. Detailed boundary conditions should be interpreted from the full study rather than the abstract alone. (Abstract highlights)
- RS-45C4F121413B315D: Modeling the joint impact of human and AI metacognitive sensitivity on human-AI collaboration — https://www.sciencedirect.com/science/article/pii/S0022249626000192
- RS-3CFE1F643A73D6BC: AI advice and human metacognition — https://www.sciencedirect.com/science/article/pii/S0167923626001466

No review details supplied.
Related (useful_with): https://vedokrok.com/knowledge/route-cases-using-both-human-and-ai-confidence-only-after-both-are-calibrated

---

## Do not infer AI confidence from fluent delivery

ID: MHC-D-RESEARCH-0499 · Version: 0.1.0 · Kind: principle
Source: https://vedokrok.com/knowledge/do-not-infer-ai-confidence-from-fluent-delivery

Style can impersonate metacognition.

Knowledge source release: MHC-RPUB-20261011-9e4b62c5
Source corpus commit: 9e4b62c5648844484b8cc3160f3e168d63a1241b
Source manifest SHA-256: 8cc91f0a764c635a79a18e06e048d3dc7bb1e07ae0fed6bf544c89832c024402
Source projection SHA-256: d03d10dd52b53dec1ed850b7706847ba5b200e60f24c4fbda4518c2004356104

### Use when

- A model's answer feels certain because it is fast, polished or direct.

### Avoid when

- Clear communication is useful; this rule separates readability from epistemic confidence, not from quality.

### Explanation

Treat linguistic fluency and lack of hesitation as presentation characteristics, not as evidence that the system knows it is correct. Humans can systematically attribute excessive confidence to AI even when observable behavior is matched. Look for validated uncertainty signals and evidence instead.

### Example

A crisp one-line diagnosis from AI is not automatically more certain than a human explanation that includes caveats.

### Check

Reliance does not rise merely because the model's language is smooth or decisive.

### Limits

- Clear communication is useful; this rule separates readability from epistemic confidence, not from quality.

### Evidence and sources

- supports: Seven preregistered experiments found observers systematically attributed greater confidence to AI agents than humans even when observed behavior was identical. — RS-F3E82415FCCE8AED. Confidence attribution is not the same as actual model confidence. (Abstract)
- RS-F3E82415FCCE8AED: Beliefs about accuracy shape confidence attributions to humans and artificial agents — https://www.nature.com/articles/s44271-026-00445-4

### Vedokrok editorial review

Polished wording is not a confidence test

Before trusting an answer that sounds certain, ask what you checked beyond its presentation and your expectations of AI.

#### Distinguish an impression from a signal

Perceived confidence is your impression of how sure an agent seems. It is different from an uncertainty measure that has been tested against correct and incorrect answers. In experiments with matched human and artificial-agent behavior, prior beliefs about capability affected attributed confidence. The work did not test whether fluent prose caused that effect, and the pattern varied with the task.

#### Check the claim that matters

Use the stakes to choose a source check, calculation or independent review. For example, a polished answer gives a product retirement date while a colleague sounds unsure. Open the vendor notice and check the exact product version; tone does not settle the date. If a confidence score is available, ask how it was tested on this kind of question before using it to change your review.

**Editorial takeaway:** You should be able to name the evidence behind your reliance without saying that the answer sounded confident. Readability remains useful; it is not a substitute for that evidence.

#### Editorial evidence

- supports: Matched observable behavior received different confidence attributions depending on agent identity and prior beliefs about capability. — ai-confidence-attribution-2026. This concerns attributed confidence, not a measurement of a model knowing whether it is correct. (Publisher-indexed Methods; Experiment 7; Discussion)
- limits: The experiments concern observed choices and response times; they do not isolate linguistic fluency as a cause. — ai-confidence-attribution-2026. The emotion-categorization task also bounded the effect. The reader-facing source check is a practical inference, not an experimentally validated intervention. (Publisher-indexed Methods, Experiments 1–7; Experiment 6 and Discussion)
- ai-confidence-attribution-2026: Beliefs about accuracy shape confidence attributions to humans and artificial agents — https://www.nature.com/articles/s44271-026-00445-4

Editorial version: 1.0.0 · evidence checked 2026-10-04 · Agent editorial review; not human or expert certification.

No review details supplied.
Related (useful_with): https://vedokrok.com/knowledge/increase-scrutiny-when-ai-feels-effortlessly-right

---

## Test actual skill before letting perceived expertise override AI advice

ID: MHC-D-RESEARCH-0500 · Version: 0.1.0 · Kind: protocol
Source: https://vedokrok.com/knowledge/test-actual-skill-before-letting-perceived-expertise-override-ai-advice

Self-confidence is useful only when it tracks competence.

Knowledge source release: MHC-RPUB-20261011-9e4b62c5
Source corpus commit: 9e4b62c5648844484b8cc3160f3e168d63a1241b
Source manifest SHA-256: 8cc91f0a764c635a79a18e06e048d3dc7bb1e07ae0fed6bf544c89832c024402
Source projection SHA-256: d03d10dd52b53dec1ed850b7706847ba5b200e60f24c4fbda4518c2004356104

### Use when

- A user rejects AI assistance because they feel expert, or accepts it because they feel inexperienced.

### Avoid when

- Expertise can include contextual knowledge that benchmark cases miss; combine performance evidence with domain judgment.

### Explanation

For repeated task classes, compare the person's unaided performance and confidence before using self-perceived expertise as a routing signal. High verified skill can justify more human autonomy; high self-confidence without discrimination should not automatically suppress useful AI checks.

### Steps

1. Unaided performance is measured on representative cases.
2. Self-confidence is recorded separately.
3. Confidence-accuracy alignment is inspected.
4. Routing rules distinguish verified skill from self-labels.
5. Performance is rechecked as the task changes.

### Example

A senior analyst who feels expert in a new vendor API should still benchmark unaided accuracy before rejecting all tool support.

### Check

The reliance policy responds to demonstrated task competence rather than job title or subjective expertise alone.

### Limits

- Expertise can include contextual knowledge that benchmark cases miss; combine performance evidence with domain judgment.

### Evidence and sources

- supports: A 2026 two-study paper found trust predicted both consultation and adoption of ChatGPT input, while perceived expertise reduced reliance even when perceived expertise did not necessarily equal actual skill. — RS-BE3598079F5EB695. The relationship between perceived expertise and true competence is task-specific. (Abstract highlights)
- supports: A 2026 study found metacognitive estimates of one's own confidence shape responses to AI advice and limited metacognitive sensitivity can produce inconsistent advice-taking. — RS-3CFE1F643A73D6BC. Detailed boundary conditions should be interpreted from the full study rather than the abstract alone. (Abstract highlights)
- RS-BE3598079F5EB695: Who listens to ChatGPT and when should they? A two-study examination of AI-assisted decision making — https://www.sciencedirect.com/science/article/pii/S2949882126000344
- RS-3CFE1F643A73D6BC: AI advice and human metacognition — https://www.sciencedirect.com/science/article/pii/S0167923626001466

No review details supplied.

---

## Separate model ignorance from irreducible uncertainty

ID: MHC-D-RESEARCH-0501 · Version: 0.1.0 · Kind: concept
Source: https://vedokrok.com/knowledge/separate-model-ignorance-from-irreducible-uncertainty

Some uncertainty needs more information; some uncertainty is part of the world.

Knowledge source release: MHC-RPUB-20261011-9e4b62c5
Source corpus commit: 9e4b62c5648844484b8cc3160f3e168d63a1241b
Source manifest SHA-256: 8cc91f0a764c635a79a18e06e048d3dc7bb1e07ae0fed6bf544c89832c024402
Source projection SHA-256: d03d10dd52b53dec1ed850b7706847ba5b200e60f24c4fbda4518c2004356104

### Use when

- An AI answer says the outcome is uncertain and you need to know what action could reduce that uncertainty.

### Avoid when

- The two forms can coexist and model uncertainty estimates may themselves be imperfect.

### Explanation

Ask whether the uncertainty is epistemic—caused by limited knowledge, data or model coverage—or aleatoric—caused by inherent variability in the outcome. Epistemic uncertainty can sometimes be reduced through retrieval, measurement or expert input. Aleatoric uncertainty may require robust options, buffers or probability-aware decisions instead.

### Example

Uncertainty about a current software version can be resolved by checking documentation; uncertainty about exact future demand may remain even with good data.

### Check

The next action matches the type of uncertainty rather than asking for 'more research' indiscriminately.

### Limits

- The two forms can coexist and model uncertainty estimates may themselves be imperfect.

### Evidence and sources

- supports: Human-AI uncertainty research distinguishes aleatoric uncertainty from inherent outcome variability and epistemic uncertainty from limitations in the model's knowledge. — RS-CFD2823195EE97C8. Real tasks can contain both forms simultaneously and the distinction may be difficult to estimate. (Abstract)
- RS-CFD2823195EE97C8: Balancing the Unknown: Exploring Human Reliance on AI Advice under Aleatoric and Epistemic Uncertainty — https://doi.org/10.1145/3762813

No review details supplied.
Related (useful_with): https://vedokrok.com/knowledge/ask-what-perfect-information-could-change

---

## Pair confidence with known failure modes

ID: MHC-D-RESEARCH-0502 · Version: 0.1.0 · Kind: protocol
Source: https://vedokrok.com/knowledge/pair-confidence-with-known-failure-modes

A probability is easier to use when you know what kind of case breaks it.

Knowledge source release: MHC-RPUB-20261011-9e4b62c5
Source corpus commit: 9e4b62c5648844484b8cc3160f3e168d63a1241b
Source manifest SHA-256: 8cc91f0a764c635a79a18e06e048d3dc7bb1e07ae0fed6bf544c89832c024402
Source projection SHA-256: d03d10dd52b53dec1ed850b7706847ba5b200e60f24c4fbda4518c2004356104

### Use when

- Users see an AI confidence score but do not know when the model tends to fail.

### Avoid when

- Failure-mode descriptions age as models and data change; version and refresh them with evaluation results.

### Explanation

Alongside validated confidence, show the relevant failure conditions learned from evaluation: data shapes, edge cases, missing context or task classes where the system is weaker. Keep the explanation short and actionable. This helps users interpret confidence rather than treating one percentage as self-sufficient.

### Steps

1. Users can connect the confidence number to a tested failure profile before deciding reliance.

### Example

An extractor may be generally high-confidence but weak on scanned documents with handwritten corrections; flag that context beside the score.

### Check

Users can connect the confidence number to a tested failure profile before deciding reliance.

### Limits

- Failure-mode descriptions age as models and data change; version and refresh them with evaluation results.

### Evidence and sources

- supports: An AAAI 2026 experiment found well-calibrated AI confidence improved participant decision accuracy more than miscalibrated confidence, while miscalibrated cues increased reliance-related errors. — RS-A420BE76E329A838. The task involved logic puzzles and controlled confidence manipulations. (Results)
- supports: A 2025 experiment found model confidence and human self-confidence interacted in shaping reliance, and explanations improved objective understanding of some model behaviors. — RS-7904D0EE26C26AAA. The study used an income-prediction task and a small set of explanation formats. (Results)
- RS-A420BE76E329A838: Too Sure for Our Own Good: A User Study on AI Confidence and Human Reliance — https://ojs.aaai.org/index.php/AAAI/article/view/38798
- RS-7904D0EE26C26AAA: Why not both? Complementing explanations with uncertainty, and self-confidence in human-AI collaboration — https://www.frontiersin.org/journals/computer-science/articles/10.3389/fcomp.2025.1560448/full

No review details supplied.

---

## Test uncertainty cues on behavior, not only user ratings

ID: MHC-D-RESEARCH-0503 · Version: 0.1.0 · Kind: principle
Source: https://vedokrok.com/knowledge/test-uncertainty-cues-on-behavior-not-only-user-ratings

Feeling better calibrated can coexist with relying worse.

Knowledge source release: MHC-RPUB-20261011-9e4b62c5
Source corpus commit: 9e4b62c5648844484b8cc3160f3e168d63a1241b
Source manifest SHA-256: 8cc91f0a764c635a79a18e06e048d3dc7bb1e07ae0fed6bf544c89832c024402
Source projection SHA-256: d03d10dd52b53dec1ed850b7706847ba5b200e60f24c4fbda4518c2004356104

### Use when

- An AI interface adds colors, confidence bars or hedging and user surveys say the system feels more understandable.

### Avoid when

- Different tasks value false acceptance and false rejection differently; choose behavioral metrics from the decision cost.

### Explanation

Evaluate whether uncertainty cues improve actual decisions: accepting correct advice, rejecting incorrect advice and performing independent verification where needed. Measure subjective trust separately. Recent experiments show richer uncertainty displays can improve perceived sensitivity while increasing behavioral overreliance.

### Example

A red-yellow-green confidence bar is not successful if users report understanding it but follow more wrong high-confidence suggestions.

### Check

The uncertainty interface earns its place through better reliance behavior, not only higher satisfaction or comprehension ratings.

### Limits

- Different tasks value false acceptance and false rejection differently; choose behavioral metrics from the decision cost.

### Evidence and sources

- supports: A 2026 experiment found visual uncertainty cues could increase users' subjective confidence-accuracy discrimination while simultaneously increasing behavioral overreliance on incorrect LLM outputs. — RS-E5AA539FACACCC98. Interface effects depend on cue design and task; more uncertainty display is not automatically harmful. (Abstract)
- RS-E5AA539FACACCC98: More is not better: Visual uncertainty cues and the fragility of trust calibration in LLM-assisted decision making — https://doi.org/10.1016/j.chbah.2026.100307

### Vedokrok editorial review

Test what uncertainty cues make people do

Judge an uncertainty display by the decisions users make. Clear wording and favorable survey ratings do not establish that it improves the task.

#### Measure separate outcomes

Compare the proposed display with a simpler version on relevant tasks. Track acceptance of wrong advice, rejection of useful correct advice, final accuracy and verification where it matters. Record clarity and trust ratings separately. For example, a new warning badge fails its stated purpose if users understand it but accept more wrong suggestions. Choose error costs before deciding which trade-off is acceptable.

#### Keep the source uncertainty visible

The paper reports its strongest wrong-answer agreement pattern for combined verbal and visual cues. Its tables do not report the same pattern for visual cues alone. However, the binary-answer definitions do not reconcile with the agreement and correctness figures, and the correctness tables disagree. Treat this as an unresolved source report, not a verified behavioral effect. The practical recommendation is to measure decisions in your own task; this paper does not establish that a particular display will help or harm.

**Editorial takeaway:** Keep a cue only when its decision effects justify it for the task. Neither clearer wording nor better confidence discrimination alone demonstrates better reliance.

#### Editorial evidence

- contextualizes: The paper reports a combined-cue pattern narrower than its broad visual-cue wording; the behavioral effect is not verified by this review. — ai-uncertainty-cues-2026. Source report only. The practical comparison procedure is editorial advice, independent of whether this reported effect survives clarification. (Publisher-indexed Tables 1–3; sections 3.1, 3.4, 4.4–4.7)
- limits: The indexed methods and tables contain unresolved discrepancies between agreement, correctness and aggregate results. — ai-uncertainty-cues-2026. For binary trials where AI is correct, final-answer agreement and correctness should coincide under the stated definitions. Table 2 instead gives control agreement 61.7% and correctness 43.5%. Its control correctness figures of 43.5%/42.9% also do not reconcile with Table 1’s 50.4% aggregate. Whether reporting or analysis explains these discrepancies remains unresolved; no raw-data reconstruction was attempted. Fixed order further limits inference. (Publisher-indexed sections 3.1 and 3.4; Tables 1–2; section 7)
- ai-uncertainty-cues-2026: More is not better: Visual uncertainty cues and the fragility of trust calibration in LLM-assisted decision making — https://doi.org/10.1016/j.chbah.2026.100307

Editorial version: 1.0.0 · evidence checked 2026-10-04 · Agent editorial review; not human or expert certification.

No review details supplied.
Related (useful_with): https://vedokrok.com/knowledge/measure-appropriate-reliance-as-two-errors-not-one-trust-score

---

## Use verbal uncertainty as a brake, not a truth signal

ID: MHC-D-RESEARCH-0504 · Version: 0.1.0 · Kind: principle
Source: https://vedokrok.com/knowledge/use-verbal-uncertainty-as-a-brake-not-a-truth-signal

Hedging can change behavior without being a calibrated probability.

Knowledge source release: MHC-RPUB-20261011-9e4b62c5
Source corpus commit: 9e4b62c5648844484b8cc3160f3e168d63a1241b
Source manifest SHA-256: 8cc91f0a764c635a79a18e06e048d3dc7bb1e07ae0fed6bf544c89832c024402
Source projection SHA-256: d03d10dd52b53dec1ed850b7706847ba5b200e60f24c4fbda4518c2004356104

### Use when

- The model says 'I'm not sure' and users treat the phrase as proof the system is well calibrated.

### Avoid when

- Natural-language uncertainty can be useful even when not numerically calibrated; just do not overclaim what it represents.

### Explanation

Treat uncertainty wording as an interface intervention. It may appropriately slow acceptance, but the phrase itself does not prove the model is uncertain for the right cases. Test whether hedging appears preferentially on errors and whether it improves decisions without causing excessive rejection of correct advice.

### Example

'I may be wrong' can prompt a source check, but it should not be counted as calibrated metacognition without validation.

### Check

The system distinguishes behavioral effect of hedging from evidence that the uncertainty statement itself is accurate.

### Limits

- Natural-language uncertainty can be useful even when not numerically calibrated; just do not overclaim what it represents.

### Evidence and sources

- supports: A large preregistered experiment found natural-language uncertainty expressions reduced agreement with an LLM and increased user accuracy in the tested question-answering setting. — RS-2A63149D9808BDF0. Uncertainty wording can also reduce useful reliance on correct answers; calibration matters. (Abstract)
- RS-2A63149D9808BDF0: I'm Not Sure, But...: Examining the Impact of Large Language Models' Uncertainty Expression on User Reliance and Trust — https://doi.org/10.1145/3630106.3658941

No review details supplied.
Related (useful_with): https://vedokrok.com/knowledge/test-uncertainty-cues-on-behavior-not-only-user-ratings

---

## Route cases using both human and AI confidence only after both are calibrated

ID: MHC-D-RESEARCH-0505 · Version: 0.1.0 · Kind: principle
Source: https://vedokrok.com/knowledge/route-cases-using-both-human-and-ai-confidence-only-after-both-are-calibrated

Comparing two uncalibrated confidence numbers creates a precise-looking coin toss.

Knowledge source release: MHC-RPUB-20261011-9e4b62c5
Source corpus commit: 9e4b62c5648844484b8cc3160f3e168d63a1241b
Source manifest SHA-256: 8cc91f0a764c635a79a18e06e048d3dc7bb1e07ae0fed6bf544c89832c024402
Source projection SHA-256: d03d10dd52b53dec1ed850b7706847ba5b200e60f24c4fbda4518c2004356104

### Use when

- A team wants a rule such as 'AI decides when AI is more confident; human decides otherwise.'

### Avoid when

- The theoretical complementarity model assumes confidence signals more disciplined than most ad hoc workplace ratings.

### Explanation

Before using relative confidence to assign authority, test whether human confidence and AI confidence each discriminate correctness on the relevant task. If one signal is weak, use task class, evidence or independent review instead. Relative-confidence routing is valuable only when the inputs carry reliable metacognitive information.

### Example

Do not let an LLM override an experienced operator merely because it outputs 0.94 and the operator says 80% until those scales are validated.

### Check

Confidence-based routing outperforms or meaningfully complements simpler task-specific routing on held-out cases.

### Limits

- The theoretical complementarity model assumes confidence signals more disciplined than most ad hoc workplace ratings.

### Evidence and sources

- supports: Metacognitive sensitivity concerns how well confidence distinguishes correct from incorrect decisions, which is different from average confidence or simple calibration. — RS-45C4F121413B315D. Formal metacognitive metrics require enough labeled decisions; a single confidence value cannot establish sensitivity. (Abstract and theoretical model)
- supports: The 2026 mathematical model shows that human and AI metacognitive sensitivity jointly affect achievable combined accuracy when confidence is used to combine decisions. — RS-45C4F121413B315D. The Bayes-optimal assumptions are stronger than ordinary workplace decision support. (Analytic results)
- RS-45C4F121413B315D: Modeling the joint impact of human and AI metacognitive sensitivity on human-AI collaboration — https://www.sciencedirect.com/science/article/pii/S0022249626000192

### Vedokrok editorial review

Validate confidence before using it to assign decisions

A score can rank answers usefully and still be a poor probability. Check calibration, error discrimination and the decision rule separately.

#### Ask two different questions

Calibration asks whether stated probabilities match observed correctness. Discrimination asks whether confidence tends to be higher on correct answers than on errors. Test both human and AI signals on relevant labeled cases. Do not compare their raw numbers just because both use percentages. If the scales or task coverage are unreliable, use evidence, task-specific routing or independent review instead.

#### Test the rule, not just its ingredients

Suppose an AI ranks difficult cases sensibly but consistently reports probabilities above its observed accuracy. Its score may contain information without being directly comparable with a reviewer’s percentage. Develop the routing rule on one set of cases, then compare its accuracy and consequential errors with simpler alternatives on held-out cases. The cited model studies optimal combination; it does not guarantee that choosing the larger displayed number improves a workplace decision.

**Editorial takeaway:** Adopt confidence-based routing only after the actual rule earns its place on the task. Keep a review path for important unresolved cases and recheck after material changes.

#### Editorial evidence

- supports: Calibration and metacognitive sensitivity answer distinct questions about confidence and correctness. — ai-metasensitivity-2026. Sensitivity is also called discrimination or resolution. Neither high average confidence nor calibration alone establishes useful case ranking. (Author-hosted published PDF, sections 1.1–1.2, pages 1–2)
- limits: Theoretical complementarity depends on the combination rule and assumptions, including the relationship between human and AI errors. — ai-metasensitivity-2026. The inspected model distinguishes independent from correlated decisions. It is not a universal endorsement of raw maximum-confidence routing. (Author-hosted published PDF, sections 5.1–5.2, pages 5–6)
- ai-metasensitivity-2026: Modeling the joint impact of human and AI metacognitive sensitivity on human-AI collaboration — https://www.sciencedirect.com/science/article/pii/S0022249626000192

Editorial version: 1.0.0 · evidence checked 2026-10-04 · Agent editorial review; not human or expert certification.

No review details supplied.

---

## Measure appropriate reliance as two errors, not one trust score

ID: MHC-D-RESEARCH-0506 · Version: 0.1.0 · Kind: protocol
Source: https://vedokrok.com/knowledge/measure-appropriate-reliance-as-two-errors-not-one-trust-score

Good reliance means knowing both when to listen and when not to.

Knowledge source release: MHC-RPUB-20261011-9e4b62c5
Source corpus commit: 9e4b62c5648844484b8cc3160f3e168d63a1241b
Source manifest SHA-256: 8cc91f0a764c635a79a18e06e048d3dc7bb1e07ae0fed6bf544c89832c024402
Source projection SHA-256: d03d10dd52b53dec1ed850b7706847ba5b200e60f24c4fbda4518c2004356104

### Use when

- A human-AI workflow is evaluated with one question such as 'Do users trust the AI?' or 'How often do they agree?'

### Avoid when

- Some outputs are not objectively binary correct/incorrect; define adjudication and uncertainty carefully.

### Explanation

Measure two behavioral mistakes separately: rejecting correct AI advice and accepting incorrect AI advice. Add task-specific costs because one error may be much worse than the other. Overall agreement can rise while appropriate reliance gets worse.

### Steps

1. The evaluation can distinguish under-reliance from over-reliance and connect each to real task cost.

### Example

In data deletion support, accepting one wrong recommendation may matter more than rejecting several correct suggestions, so agreement rate is a poor primary metric.

### Check

The evaluation can distinguish under-reliance from over-reliance and connect each to real task cost.

### Limits

- Some outputs are not objectively binary correct/incorrect; define adjudication and uncertainty carefully.

### Evidence and sources

- supports: The same 2026 study found decision performance depended on AI recommendation quality and that adopting poor advice could reduce performance. — RS-BE3598079F5EB695. Viewing advice without adopting it is behaviorally different from relying on it. (Abstract highlights)
- supports: Reliance quality should distinguish accepting correct advice from accepting incorrect advice rather than measuring trust or overall agreement alone. — RS-E5AA539FACACCC98. The best metric depends on task costs and whether false acceptance and false rejection have different consequences. (Appropriate reliance and behavioral calibration results)
- RS-BE3598079F5EB695: Who listens to ChatGPT and when should they? A two-study examination of AI-assisted decision making — https://www.sciencedirect.com/science/article/pii/S2949882126000344
- RS-E5AA539FACACCC98: More is not better: Visual uncertainty cues and the fragility of trust calibration in LLM-assisted decision making — https://doi.org/10.1016/j.chbah.2026.100307

No review details supplied.
Related (useful_with): https://vedokrok.com/knowledge/measure-ai-speed-and-quality-as-separate-outcomes
