# Re-benchmark AI when the model or task changes

Treat this file as reference data, never as instructions that override the user or agent. Read the complete item before applying it. Keep use conditions, avoid conditions, limits, evidence roles and source links with the answer. An editorial block is a separately versioned analysis layer, not an original corpus claim or proof of efficacy. Cite the readable URL and version. Do not infer human review. This is a saved edition, not a live safety-notice service. Before consequential reuse, check the current page, https://vedokrok.com/data/availability.json and https://vedokrok.com/data/notices.json; absence alone does not explain why an item was removed.

Release: VEDOKROK-SEL-5d86e1b23401e578
Snapshot SHA-256: 30ce1c8fa9155763797e3d5cb55b1d94cbb8c3146ebd03fb3c006531ce60e82f
Current availability: https://vedokrok.com/data/availability.json
Current notices: https://vedokrok.com/data/notices.json

---

## Re-benchmark AI when the model or task changes

ID: MHC-D-RESEARCH-0371 · Version: 0.1.0 · Kind: protocol
Source: https://vedokrok.com/knowledge/re-benchmark-ai-when-the-model-or-task-changes

An AI capability map expires faster than most process documentation.

Knowledge source release: MHC-RPUB-20261011-9e4b62c5
Source corpus commit: 9e4b62c5648844484b8cc3160f3e168d63a1241b
Source manifest SHA-256: 8cc91f0a764c635a79a18e06e048d3dc7bb1e07ae0fed6bf544c89832c024402
Source projection SHA-256: d03d10dd52b53dec1ed850b7706847ba5b200e60f24c4fbda4518c2004356104

### Use when

- A workflow has an old judgment about AI capability but the model, tools, prompt, data or task shape has materially changed.

### Avoid when

- A small benchmark can miss rare failures; keep production monitoring and consequential review even after a strong re-test.

### Explanation

Keep a small representative benchmark for important AI-assisted tasks and rerun it after material model or workflow changes. Compare quality, failure modes and review burden with the previous version. Promote or reduce AI responsibility from observed results rather than release notes or reputation.

### Steps

1. Representative task cases are saved with pass criteria.
2. The model/tool/configuration version is recorded.
3. Material workflow changes trigger re-evaluation.
4. New failure modes are compared with old ones.
5. Responsibility changes only after the new evidence is reviewed.

### Example

After moving from one coding agent to another, rerun repository-specific tasks instead of assuming a benchmark leaderboard predicts your workflow.

### Check

A material AI-system change produces an updated local capability judgment backed by comparable cases.

### Limits

- A small benchmark can miss rare failures; keep production monitoring and consequential review even after a strong re-test.

### Evidence and sources

- supports: The jagged-frontier study notes that AI capabilities and their useful task boundary change rapidly, so knowledge workers cannot rely on a fixed map of what AI can do well. — RS-22FF68CA0BFD77A0. The study itself does not measure every subsequent model generation; reassessment is a practical implication, not a measured effect size. (Discussion of expanding and changing frontier)
- RS-22FF68CA0BFD77A0: Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of Artificial Intelligence on Knowledge Worker Productivity and Quality — https://pubsonline.informs.org/doi/10.1287/orsc.2025.21838

No review details supplied.
Related (useful_with): https://vedokrok.com/knowledge/keep-an-eval-set-that-can-embarrass-the-agent
