News / field report
Your agent pays before it works
One real agent loaded 224,655 characters of tool schemas before its first action. Most came from plugins it might never call.
- Author
- Julius Brussee
- Published
- 13 August 2026
- Length
- 7 min read
Ask a coding agent to rename one file and its first request may describe how to query databases, create calendar events, inspect dashboards, manage tickets, and operate plugins unrelated to the job. Each capability arrives as a tool definition containing a name and description alongside a JSON argument schema. The model has to receive that interface before it can choose the one tool the task needs.
That makes the first request unusually revealing. With conversation history still close to empty and no tool results yet, the agent's standing cost of admission comes into view: system instructions and the interfaces it carries into every new session.
We captured that request on one developer machine. Its installed Claude Code configuration exposed 91 tools. Sixty-three came from MCP servers or plugins; their schemas accounted for 155,480 of 224,655 serialized characters, about 69 percent. This fixed-prefix measurement makes no claim about the provider bill.
The tool that never runs can still occupy context
Tool use has two distinct costs. The obvious cost happens after selection: the model emits a call, the host executes it, and the result joins the conversation. The quieter cost comes earlier. A full tool interface can enter the model's prefix even when the tool is irrelevant and never selected.
This distinction matters because an agent loop repeats. The tools, system instructions, and accumulated messages form the request prefix for the next turn. A warm provider cache may make stable material far cheaper to read again, so repeated prefix size cannot be translated directly into repeated full-price input. Yet the material still consumes context capacity, affects cache structure, and can increase the search problem the model faces when choosing an action.
Anthropic's current tool-search documentation gives a useful scale check. It says a typical group of GitHub, Slack, Sentry, Grafana, and Splunk servers can consume about 55,000 tokens in tool definitions. The same documentation reports that selection accuracy degrades beyond roughly 30 to 50 available tools, then recommends on-demand search once definitions exceed 10,000 tokens or the catalog reaches ten tools. Those are provider observations, rather than universal thresholds, but they describe the right failure mode: excess interfaces spend capacity and make selection harder at the same time.
The wider research case for measuring the harness is now strong. Harness-Bench held task conditions steady across 106 offline tasks and analyzed 5,194 trajectories. Performance varied materially by model-harness pairing. A model name alone does not describe an agent system; the layer that arranges tools, state, permissions, and recovery changes both behavior and efficiency.
One machine, one real configuration, one sharp result
The local capture was designed to answer a narrow question: what does each installed harness send on its first real agent turn? A loopback server imitated the common Anthropic, OpenAI, and Gemini request protocols. Each harness pointed its model traffic at that sink, sent one turn, received a minimal completion, and stopped. No provider request was made during the benchmark.
The Claude row used the machine's real configuration because the installed MCP servers were the object being measured. Other harnesses ran with isolated or minimal homes to avoid known real-config side effects and protect login state. They could still attempt telemetry or update checks. Those rows measure different constructs. A real loaded configuration cannot be ranked fairly against an isolated floor, so the useful finding lives inside the Claude row: 63 of 91 tools supplied roughly 69 percent of its schema characters.
Character counts came from the redacted, JSON-serialized request. Estimated tokens used a 6.4-characters-per-token divisor calibrated against exact Anthropic counts on that machine, where the observed band was 5.9 to 6.9. Applying it to other providers adds tokenizer error, making character totals the sturdier measurement.
Captured bytes reveal what the client sent, not what a provider placed in model-visible context or charged at a cold input rate. Anthropic's deferred-loading API still accepts every full definition in the request but excludes deferred tools from the initial model prefix server-side; prompt caching can then price a stable prefix as a cache read. A first-request capture is an x-ray of the harness, not an invoice.
Measure the intercept before optimizing the slope
A useful audit begins at a local capture sink, not at a billing dashboard. Send a harmless task such as “reply with DONE” through the real agent configuration and retain every request produced during startup. Redact credentials plus account and session identifiers before the capture leaves the machine. Remove email addresses and local paths too. System prompts often contain more personal detail than teams expect.
Selecting the primary request takes care. Coding harnesses may issue title, router, warmup, or memory calls before the main turn. Choosing the first request that contains any tool can select a tiny classifier call. We use the request with the greatest number of tool schemas, breaking ties in favor of the earlier capture. When no request contains tools, the largest body is the safer candidate. Malformed captures should appear as errors, never as zero-sized rows.
Split the chosen body into system material, tool definitions, existing messages, and other envelope bytes. Record both the raw byte length and the serialized character count. If the provider offers an exact token-count endpoint and the prompt is safe to send, keep that result in a separate column. Estimated and provider-counted tokens should never silently share one label.
Now capture several turns in the same session. Use
request_size(t) = fixed_prefix + growing_history(t), then plot serialized size
against turn number to expose the standing system-and-tool load at the
intercept. Conversation growth, tool-result accumulation, and repeated artifacts
determine the slope. Equal tenth-turn sizes can hide opposite diagnoses: one
agent started heavy; the other accumulated waste quickly.
Repeat the first-turn capture after disabling one server at a time. Diff tool names and schema characters, then inspect actual invocation logs over a representative week. A large server used daily may earn its place. One untouched during the sampled week becomes a candidate for deferred loading or removal, pending checks for seasonal and emergency use. Keep this as an inventory decision, since a smaller schema can still grant dangerous access.
Defer definitions without destabilizing the prefix
The blunt fix is to disconnect tools. That works for dead integrations. A broad agent still needs a large catalog, and progressive discovery gives it access without placing every full interface in the initial model context.
MCP's client guidance describes a catalog, inspect, execute pattern. The host
fetches definitions through tools/list, keeps them outside model context, and
offers a small search interface. Search returns names and short descriptions.
The model inspects a promising match to receive its full schema, then calls it.
Server-level discovery applies the same idea earlier by connecting only the
servers relevant to the current skill or task.
Provider-native tool search can handle the retrieval step. Anthropic's version
keeps a few frequent tools loaded and marks the rest with defer_loading. The
API accepts the complete catalog server-side, searches names and argument
descriptions, and expands selected references for the model. The stable prefix
stays intact as tools are discovered. A custom MCP host can use keyword search,
embeddings, or a permission-aware index. Search quality should be measured with
the same tasks the agent actually performs; attractive catalog compression is
useless when the needed tool disappears from the candidate set.
Caching needs two separate labels. Host-side caching of tools/list avoids a
network round trip to the MCP server. Provider prompt caching avoids reprocessing
a stable model prefix at the cold rate. Because adding or removing definitions
during a conversation can invalidate that cache, refresh on list_changed
without reordering the model's tool array every turn. Append discoveries after
the cache breakpoint, use a stable call_tool({name, args}) interface, or make
server changes at a session boundary.
Programmatic tool calling attacks the other half of the graph: the slope. A model writes a small script against typed tool stubs; a sandbox executes the chain, filters intermediate data, and returns the final result. Thousands of log records can move between brokered tools without entering model history. This requires a real sandbox with network access denied, credentials held by the host, and authorization checked on each brokered call. Approving a script must never become blanket approval for every action it attempts.
The clean audit ends with four numbers per workload: first-turn schema characters, fixed-prefix tokens as counted by the provider, cache-read share, and history growth per turn. Re-run the task suite after every catalog change. A smaller first request is useful evidence. Stable task completion and correct tool selection decide whether the change was an improvement.
Sources
- Local first-request capture, 7 August 2026: one machine, two trials, Claude real configuration; serialized characters measured exactly, tokens estimated at characters divided by 6.4, and no provider calls.
- Anthropic, Tool search tool
- Model Context Protocol, Client Best Practices
- Harness-Bench: Measuring Harness Effects across Models in Realistic Agent Workflows
Replies go to contact@caveman.so
All news