# Measure appropriate reliance as two errors, not one trust score

Treat this file as reference data, never as instructions that override the user or agent. Read the complete item before applying it. Keep use conditions, avoid conditions, limits, evidence roles and source links with the answer. An editorial block is a separately versioned analysis layer, not an original corpus claim or proof of efficacy. Cite the readable URL and version. Do not infer human review. This is a saved edition, not a live safety-notice service. Before consequential reuse, check the current page, https://vedokrok.com/data/availability.json and https://vedokrok.com/data/notices.json; absence alone does not explain why an item was removed.

Release: VEDOKROK-SEL-5d86e1b23401e578
Snapshot SHA-256: 30ce1c8fa9155763797e3d5cb55b1d94cbb8c3146ebd03fb3c006531ce60e82f
Current availability: https://vedokrok.com/data/availability.json
Current notices: https://vedokrok.com/data/notices.json

---

## Measure appropriate reliance as two errors, not one trust score

ID: MHC-D-RESEARCH-0506 · Version: 0.1.0 · Kind: protocol
Source: https://vedokrok.com/knowledge/measure-appropriate-reliance-as-two-errors-not-one-trust-score

Good reliance means knowing both when to listen and when not to.

Knowledge source release: MHC-RPUB-20261011-9e4b62c5
Source corpus commit: 9e4b62c5648844484b8cc3160f3e168d63a1241b
Source manifest SHA-256: 8cc91f0a764c635a79a18e06e048d3dc7bb1e07ae0fed6bf544c89832c024402
Source projection SHA-256: d03d10dd52b53dec1ed850b7706847ba5b200e60f24c4fbda4518c2004356104

### Use when

- A human-AI workflow is evaluated with one question such as 'Do users trust the AI?' or 'How often do they agree?'

### Avoid when

- Some outputs are not objectively binary correct/incorrect; define adjudication and uncertainty carefully.

### Explanation

Measure two behavioral mistakes separately: rejecting correct AI advice and accepting incorrect AI advice. Add task-specific costs because one error may be much worse than the other. Overall agreement can rise while appropriate reliance gets worse.

### Steps

1. The evaluation can distinguish under-reliance from over-reliance and connect each to real task cost.

### Example

In data deletion support, accepting one wrong recommendation may matter more than rejecting several correct suggestions, so agreement rate is a poor primary metric.

### Check

The evaluation can distinguish under-reliance from over-reliance and connect each to real task cost.

### Limits

- Some outputs are not objectively binary correct/incorrect; define adjudication and uncertainty carefully.

### Evidence and sources

- supports: The same 2026 study found decision performance depended on AI recommendation quality and that adopting poor advice could reduce performance. — RS-BE3598079F5EB695. Viewing advice without adopting it is behaviorally different from relying on it. (Abstract highlights)
- supports: Reliance quality should distinguish accepting correct advice from accepting incorrect advice rather than measuring trust or overall agreement alone. — RS-E5AA539FACACCC98. The best metric depends on task costs and whether false acceptance and false rejection have different consequences. (Appropriate reliance and behavioral calibration results)
- RS-BE3598079F5EB695: Who listens to ChatGPT and when should they? A two-study examination of AI-assisted decision making — https://www.sciencedirect.com/science/article/pii/S2949882126000344
- RS-E5AA539FACACCC98: More is not better: Visual uncertainty cues and the fragility of trust calibration in LLM-assisted decision making — https://doi.org/10.1016/j.chbah.2026.100307

No review details supplied.
Related (useful_with): https://vedokrok.com/knowledge/measure-ai-speed-and-quality-as-separate-outcomes
