# Combine grader types instead of asking one LLM judge to decide everything

Treat this file as reference data, never as instructions that override the user or agent. Read the complete item before applying it. Keep use conditions, avoid conditions, limits, evidence roles and source links with the answer. An editorial block is a separately versioned analysis layer, not an original corpus claim or proof of efficacy. Cite the readable URL and version. Do not infer human review. This is a saved edition, not a live safety-notice service. Before consequential reuse, check the current page, https://vedokrok.com/data/availability.json and https://vedokrok.com/data/notices.json; absence alone does not explain why an item was removed.

Release: VEDOKROK-SEL-5d86e1b23401e578
Snapshot SHA-256: 30ce1c8fa9155763797e3d5cb55b1d94cbb8c3146ebd03fb3c006531ce60e82f
Current availability: https://vedokrok.com/data/availability.json
Current notices: https://vedokrok.com/data/notices.json

---

## Combine grader types instead of asking one LLM judge to decide everything

ID: MHC-D-RESEARCH-0957 · Version: 0.1.0 · Kind: protocol
Source: https://vedokrok.com/knowledge/combine-grader-types-instead-of-asking-one-llm-judge-to-decide-everything

A flexible judge is useful; a monoculture of judgment is fragile.

Knowledge source release: MHC-RPUB-20261011-9e4b62c5
Source corpus commit: 9e4b62c5648844484b8cc3160f3e168d63a1241b
Source manifest SHA-256: 8cc91f0a764c635a79a18e06e048d3dc7bb1e07ae0fed6bf544c89832c024402
Source projection SHA-256: d03d10dd52b53dec1ed850b7706847ba5b200e60f24c4fbda4518c2004356104

### Use when

- An open-ended agent task is scored entirely by one model-based evaluator.

### Avoid when

- More graders can add cost and conflicting signals; each must serve a clear decision.

### Explanation

Use deterministic checks where the outcome is objective, model-based rubrics for nuanced qualities, and human review to calibrate subjective grading. Design each grader for a specific dimension rather than one omnibus 'good/bad' score. Recalibrate model graders when tasks or models change.

### Steps

1. Use code/tests for objective conditions.
2. Use model rubrics for open-ended qualities.
3. Use human examples to calibrate subjective graders.
4. Keep grader dimensions separate enough to debug disagreement.

### Example

A research agent can be graded by source existence checks, citation-groundedness rubric, coverage checks and expert spot review.

### Check

No single probabilistic grader has unchecked authority over every success dimension.

### Limits

- More graders can add cost and conflicting signals; each must serve a clear decision.

### Evidence and sources

- supports: Anthropic's 2026 agent-eval guidance recommends combining code-based, model-based and human graders according to task needs. — RS-EE9379CE3D2ABBDD. More graders can add cost and conflicting signals; each must serve a clear decision. (See source record)
- RS-EE9379CE3D2ABBDD: Demystifying evals for AI agents — https://www.anthropic.com/engineering/demystifying-evals-for-ai-agents

No review details supplied.
Related (useful_with): https://vedokrok.com/knowledge/measure-agreement-on-the-criteria-not-only-the-verdict
