# Report how much work an abstaining system actually covers

Treat this file as reference data, never as instructions that override the user or agent. Read the complete item before applying it. Keep use conditions, avoid conditions, limits, evidence roles and source links with the answer. An editorial block is a separately versioned analysis layer, not an original corpus claim or proof of efficacy. Cite the readable URL and version. Do not infer human review. This is a saved edition, not a live safety-notice service. Before consequential reuse, check the current page, https://vedokrok.com/data/availability.json and https://vedokrok.com/data/notices.json; absence alone does not explain why an item was removed.

Release: VEDOKROK-SEL-5d86e1b23401e578
Snapshot SHA-256: 30ce1c8fa9155763797e3d5cb55b1d94cbb8c3146ebd03fb3c006531ce60e82f
Current availability: https://vedokrok.com/data/availability.json
Current notices: https://vedokrok.com/data/notices.json

---

## Report how much work an abstaining system actually covers

ID: MHC-D-RESEARCH-0992 · Version: 0.1.0 · Kind: checklist
Source: https://vedokrok.com/knowledge/report-how-much-work-an-abstaining-system-actually-covers

Being right on the easiest fifth is not the same as handling the whole queue.

Knowledge source release: MHC-RPUB-20261011-9e4b62c5
Source corpus commit: 9e4b62c5648844484b8cc3160f3e168d63a1241b
Source manifest SHA-256: 8cc91f0a764c635a79a18e06e048d3dc7bb1e07ae0fed6bf544c89832c024402
Source projection SHA-256: d03d10dd52b53dec1ed850b7706847ba5b200e60f24c4fbda4518c2004356104

### Use when

- An AI workflow sends uncertain cases to a human and reports high accuracy on what remains.

### Avoid when

- Raw model confidence is not a validated probability. Evidence from selective classifiers does not provide an automatic safety guarantee for a language-model workflow.

### Explanation

Measure both the quality of accepted answers and coverage: the share of incoming cases the system handles. Track the rejected cases and the human work they create. Choose the operating point from observed trade-offs, not from the model's confident tone.

### Checklist

- Define what counts as acceptance, abstention and a correct result.
- Report accepted-case errors alongside accepted share and review workload.
- Check whether difficult groups are disproportionately left for people.

### Example

An extractor handles 70 of 100 documents and refers 30. Its accuracy on the 70 must not be reported as accuracy on all 100.

### Check

The report makes accepted coverage, residual errors and referral load visible together.

### Limits

- Raw model confidence is not a validated probability. Evidence from selective classifiers does not provide an automatic safety guarantee for a language-model workflow.

### Evidence and sources

- supports: Selective classification can trade coverage for lower risk on the cases it accepts, as demonstrated in the cited image-classification setting. — RS-700A3E0A4271B66C. No guarantee transfers to an uncalibrated language-model confidence statement or a new population. (Abstract)
- RS-700A3E0A4271B66C: Selective Classification for Deep Neural Networks — https://arxiv.org/abs/1705.08500

No review details supplied.
Related (useful_with): https://vedokrok.com/knowledge/judge-the-expensive-error-separately-from-the-common-one
