Protocol
Turn a verified AI failure into a regression test
A painful failure should pay rent the second time.
When it fits
- You discover an AI output or agent trajectory that failed in a way you care about.
When to avoid it
- Known-failure tests do not cover novel failures; keep adding fresh tasks and production monitoring.
Why it matters
After fixing the immediate problem, preserve a minimal representative task that reproduces the failure and a grader that detects it. Add it to the relevant eval suite before changing prompts, skills or models again. Tag the failure class so later regressions can be diagnosed rather than hidden inside one aggregate score.
Steps
- The old failure would be caught automatically if it returned tomorrow.
An example
If an agent silently edits an unrelated configuration file, add a task whose grader verifies both requested behavior and untouched-file constraints.
Check your result
The old failure would be caught automatically if it returned tomorrow.
Keep this limit in mind
- Known-failure tests do not cover novel failures; keep adding fresh tasks and production monitoring.
Connected ideas
Useful withKeep an eval set that can embarrass the agent
Evidence and sources
Anthropic recommends sourcing eval tasks from real failures and keeping regression suites so fixes can be tested against previously observed problems.
A regression suite covers known failures and must be supplemented with new, harder and distribution-shifted tasks.
Demystifying evals for AI agents · Collect tasks and maintain eval suites