Protocol
Retrieve tool schemas just in time
A tool can exist without occupying attention on every turn.
Before you use it
- Use when
- An agent has access to a large tool catalog and every request carries all tool descriptions into the model context.
Keep this limit in mind
- A router can hide the right tool. Optimize for end-task quality and safe fallback, not minimum schema count.
An idea to explore.
Try this idea.
Inspect the example, try the idea in your situation, then check your next move.
Why it matters
Index tool descriptions and retrieve a small candidate set from the current intent. Present only those schemas as callable tools, while preserving a fallback when retrieval confidence is weak. Evaluate routing recall, final task success, latency and context cost together.
Steps
- Index tools by the jobs and entities they operate on.
- Retrieve a bounded candidate set from the current request.
- Expose only those schemas to the model for the turn.
- Fall back to broader discovery when the router is uncertain.
- Measure missed-tool failures as well as cost and latency.
An example
A finance question retrieves account and transaction tools instead of loading schemas for HR, calendar, CRM and deployment tools into the same request.
Check your result
The system can report tool-retrieval recall and task success, not only the tokens saved by showing fewer schemas.
Save and practise this idea
My learning
Read. Recall. Repeat.
Save this idea and return to it on any device.
Keep exploring
Connected ideas
You are readingRetrieve tool schemas just in time
More from this collection
Operate AI agents as systems you can replay, verify and constrain
Measure the active tool set, not the catalog sizeEvidence and sources
Large tool catalogs can be routed semantically so only a request-relevant subset of tool schemas is placed in the model's active tool context.
Routing can hide the correct tool. Recall and fallback behavior must be evaluated alongside token cost, latency and tool-selection accuracy.
The 100-Tool Agent Is a Trap: Scaling with Semantic Routers and JIT Context · 9:19-24:27, retrieve tools, load schemas just in time, evaluate top-K and recover from misses
Edition and review
Version 0.1.0
No review details are supplied for this edition.