Protocol
Retrieve tool schemas just in time
A tool can exist without occupying attention on every turn.
When it fits
- An agent has access to a large tool catalog and every request carries all tool descriptions into the model context.
When to avoid it
- A router can hide the right tool. Optimize for end-task quality and safe fallback, not minimum schema count.
Why it matters
Index tool descriptions and retrieve a small candidate set from the current intent. Present only those schemas as callable tools, while preserving a fallback when retrieval confidence is weak. Evaluate routing recall, final task success, latency and context cost together.
Steps
- Index tools by the jobs and entities they operate on.
- Retrieve a bounded candidate set from the current request.
- Expose only those schemas to the model for the turn.
- Fall back to broader discovery when the router is uncertain.
- Measure missed-tool failures as well as cost and latency.
An example
A finance question retrieves account and transaction tools instead of loading schemas for HR, calendar, CRM and deployment tools into the same request.
Check your result
The system can report tool-retrieval recall and task success, not only the tokens saved by showing fewer schemas.
Keep this limit in mind
- A router can hide the right tool. Optimize for end-task quality and safe fallback, not minimum schema count.
Connected ideas
Useful withUse progressive disclosure for agent instructions
Evidence and sources
Large tool catalogs can be routed semantically so only a request-relevant subset of tool schemas is placed in the model's active tool context.
Routing can hide the correct tool. Recall and fallback behavior must be evaluated alongside token cost, latency and tool-selection accuracy.
The 100-Tool Agent Is a Trap: Scaling with Semantic Routers and JIT Context · 9:19-24:27, retrieve tools, load schemas just in time, evaluate top-K and recover from misses