Protocol

Retrieve tool schemas just in time

A tool can exist without occupying attention on every turn.

About 1 min read

Before you use it

Use when
  • An agent has access to a large tool catalog and every request carries all tool descriptions into the model context.

Keep this limit in mind

  • A router can hide the right tool. Optimize for end-task quality and safe fallback, not minimum schema count.

An idea to explore.

Try this idea.

Inspect the example, try the idea in your situation, then check your next move.

Keep reading

Why it matters

Index tool descriptions and retrieve a small candidate set from the current intent. Present only those schemas as callable tools, while preserving a fallback when retrieval confidence is weak. Evaluate routing recall, final task success, latency and context cost together.

Steps

  1. Index tools by the jobs and entities they operate on.
  2. Retrieve a bounded candidate set from the current request.
  3. Expose only those schemas to the model for the turn.
  4. Fall back to broader discovery when the router is uncertain.
  5. Measure missed-tool failures as well as cost and latency.

An example

A finance question retrieves account and transaction tools instead of loading schemas for HR, calendar, CRM and deployment tools into the same request.

Check your result

The system can report tool-retrieval recall and task success, not only the tokens saved by showing fewer schemas.

Save and practise this idea

My learning

Read. Recall. Repeat.

Save this idea and return to it on any device.

Use email
Keep exploring

Connected ideas

You are readingRetrieve tool schemas just in time

More from this collection

Operate AI agents as systems you can replay, verify and constrain

Measure the active tool set, not the catalog size
Evidence and sources
Supports

Large tool catalogs can be routed semantically so only a request-relevant subset of tool schemas is placed in the model's active tool context.

Routing can hide the correct tool. Recall and fallback behavior must be evaluated alongside token cost, latency and tool-selection accuracy.

The 100-Tool Agent Is a Trap: Scaling with Semantic Routers and JIT Context · 9:19-24:27, retrieve tools, load schemas just in time, evaluate top-K and recover from misses

All sources (1)
Edition and review

Version 0.1.0

No review details are supplied for this edition.

Read this idea as JSON · Release details