Protocol
Replay from the failure checkpoint, not from memory
A reproducible failure is more useful than a fresh attempt that happens to succeed.
When it fits
- An agent failed late in a long run and a full rerun is expensive, slow or unlikely to recreate the same decision.
When to avoid it
- A checkpoint is only as good as the state it captures. External systems, time and hidden side effects can still prevent an exact reproduction.
Why it matters
Checkpoint meaningful execution state before expensive or consequential boundaries. When a failure appears, restore the nearest useful checkpoint and change one factor: the model, a tool response, a policy or a piece of state. Compare the resulting decision with the original path instead of asking the system to recreate the whole past from scratch.
Steps
- Choose a checkpoint before the decision you want to inspect.
- Restore the captured model, tool and task state.
- Change one intervention while holding the rest as stable as practical.
- Compare both the next decision and the final outcome.
An example
A support agent issued the wrong refund after seven tool calls. Replay from the checkpoint before the refund decision with the repaired policy tool rather than rerunning the entire conversation.
Check your result
You can name what was held constant, what changed and whether the decision or outcome changed after replay.
Keep this limit in mind
- A checkpoint is only as good as the state it captures. External systems, time and hidden side effects can still prevent an exact reproduction.
Connected ideas
Useful withCheckpoint meaningful progress before a long agent crosses a fragile boundary
Evidence and sources
A checkpointed agent run can be replayed from an intermediate state while changing one model or tool decision, which can make interventions easier to compare than full reruns.
Replay quality depends on what state the checkpoint actually captures; hidden external state can still make the reproduced execution differ from production.
Your Agents Need a Save Button · 2:03-10:54, production checkpoints, change one execution part and apply an intervention across a cohort