Protocol
Evaluate what memory retrieves, not just what it stores
A perfect archive can still produce a bad working memory.
Before you use it
- Use when
- A memory-enabled agent still misses old findings or fills context with irrelevant notes.
Keep this limit in mind
- Recall targets can overfit known questions; periodically add new tasks and audit whether the retrieval policy generalizes.
An idea to explore.
Try this idea.
Inspect the example, try the idea in your situation, then check your next move.
Why it matters
Create recall tests from real tasks: which stored items should appear, which should stay out, and how high the important item should rank. Measure misses and distracting over-recall separately. When a task fails, inspect whether the needed fact was absent from storage, retrieved poorly or ignored after retrieval; those are different failures.
Steps
- Memory quality has a task-level recall and noise test, not only a count of saved items.
An example
For a recurring customer migration, test whether the agent retrieves the current mapping exception without also loading months of unrelated status notes.
Check your result
Memory quality has a task-level recall and noise test, not only a count of saved items.
Save and practise this idea
My learning
Read. Recall. Repeat.
Save this idea and return to it on any device.
Keep exploring
Connected ideas
You are readingEvaluate what memory retrieves, not just what it stores
Ideas that link here
These ideas link to this one. Check their conditions and limits before use.
Use before this ideaGive agent memory a write-manage-read policy
Evidence and sources
The same memory-harness work argues that retrieval ranking and recall policy can determine whether stored memory helps or wastes context.
Results come from a particular research-agent experiment and should be re-tested locally.
Memory Harnesses for Long-Running Research Agents · Bad memory spends tokens in the wrong direction; recall policy as evaluation target
All sources (1)
Edition and review
Version 0.1.0
No review details are supplied for this edition.