Principle
Retrieve context just in time instead of preloading the whole project
Relevant context is valuable; irrelevant context taxes every token after it.
When it fits
- An agent starts every task with huge documentation, memory and historical transcripts loaded into context.
When to avoid it
- Just-in-time retrieval fails if indexing is weak or the agent does not know important information exists.
Why it matters
Load the stable rules and task essentials up front, then retrieve project details, documents or memory when the current step needs them. This reduces context pollution and keeps attention focused. Add retrieval cues or source indexes so the agent knows what can be fetched later.
An example
Load repository rules and task goals at start; fetch a specific design document only when modifying that component.
Check your result
Most loaded context is relevant to the current stage rather than merely potentially useful someday.
Keep this limit in mind
- Just-in-time retrieval fails if indexing is weak or the agent does not know important information exists.
Evidence and sources
Supports
Anthropic's context-engineering guidance recommends treating context as finite and using just-in-time retrieval instead of indiscriminate preloading.
Just-in-time retrieval fails if indexing is weak or the agent does not know important information exists.
Effective context engineering for AI agents · See source record