---
title: "Your agent pays before it works"
description: "Ask a coding agent to rename one file and its first request may describe how to"
canonical: https://caveman.so/news/your-agent-pays-before-it-works
last-updated: 2026-08-13
---

# Your agent pays before it works

Ask a coding agent to rename one file and its first request may describe how to
query databases, create calendar events, inspect dashboards, manage tickets, and
operate plugins unrelated to the job. Each capability arrives as a
tool definition containing a name and description alongside a JSON argument
schema. The model has to receive that interface before it can choose the one
tool the task needs.

That makes the first request unusually revealing. With conversation history
still close to empty and no tool results yet, the agent's standing cost of
admission comes into view: system instructions and the interfaces it carries
into every new session.

We captured that request on one developer machine. Its installed Claude Code
configuration exposed 91 tools. Sixty-three came from MCP servers or plugins;
their schemas accounted for 155,480 of 224,655 serialized characters, about 69
percent. This fixed-prefix measurement makes no claim about the provider bill.

## The tool that never runs can still occupy context

Tool use has two distinct costs. The obvious cost happens after selection: the
model emits a call, the host executes it, and the result joins the conversation.
The quieter cost comes earlier. A full tool interface can enter the model's
prefix even when the tool is irrelevant and never selected.

This distinction matters because an agent loop repeats. The tools, system
instructions, and accumulated messages form the request prefix for the next
turn. A warm provider cache may make stable material far cheaper to read again,
so repeated prefix size cannot be translated directly into repeated full-price
input. Yet the material still consumes context capacity, affects cache
structure, and can increase the search problem the model faces when choosing an
action.

Anthropic's current tool-search documentation gives a useful scale check. It
says a typical group of GitHub, Slack, Sentry, Grafana, and Splunk servers can
consume about 55,000 tokens in tool definitions. The same documentation reports
that selection accuracy degrades beyond roughly 30 to 50 available tools, then
recommends on-demand search once definitions exceed 10,000 tokens or the catalog
reaches ten tools. Those are provider observations, rather than universal
thresholds, but they describe the right failure mode: excess interfaces spend
capacity and make selection harder at the same time.

The wider research case for measuring the harness is now strong. Harness-Bench
held task conditions steady across 106 offline tasks and analyzed 5,194
trajectories. Performance varied materially by model-harness pairing. A model
name alone does not describe an agent system; the layer that arranges tools,
state, permissions, and recovery changes both behavior and efficiency.

## One machine, one real configuration, one sharp result

The local capture was designed to answer a narrow question: what does each
installed harness send on its first real agent turn? A loopback server imitated
the common Anthropic, OpenAI, and Gemini request protocols. Each harness pointed
its model traffic at that sink, sent one turn, received a minimal completion,
and stopped. No provider request was made during the benchmark.

The Claude row used the machine's real configuration because the installed MCP
servers were the object being measured. Other harnesses ran with isolated or
minimal homes to avoid known real-config side effects and protect login state.
They could still attempt telemetry or update checks. Those rows measure
different constructs. A real loaded configuration cannot be ranked fairly
against an isolated floor, so the useful finding lives inside the Claude row:
63 of 91 tools supplied roughly 69 percent of its schema characters.

Character counts came from the redacted, JSON-serialized request. Estimated
tokens used a 6.4-characters-per-token divisor calibrated against exact
Anthropic counts on that machine, where the observed band was 5.9 to 6.9.
Applying it to other providers adds tokenizer error, making character totals
the sturdier measurement.

Captured bytes reveal what the client sent, not what a provider placed in
model-visible context or charged at a cold input rate. Anthropic's
deferred-loading API still accepts every full definition in the request but
excludes deferred tools from the initial model prefix server-side; prompt
caching can then price a stable prefix as a cache read. A first-request capture
is an x-ray of the harness, not an invoice.

## Measure the intercept before optimizing the slope

A useful audit begins at a local capture sink, not at a billing dashboard. Send
a harmless task such as “reply with DONE” through the real agent configuration
and retain every request produced during startup. Redact credentials plus
account and session identifiers before the capture leaves the machine. Remove
email addresses and local paths too. System prompts often contain more personal
detail than teams expect.

Selecting the primary request takes care. Coding harnesses may issue title,
router, warmup, or memory calls before the main turn. Choosing the first request
that contains any tool can select a tiny classifier call. We use the request
with the greatest number of tool schemas, breaking ties in favor of the earlier
capture. When no request contains tools, the largest body is the safer candidate.
Malformed captures should appear as errors, never as zero-sized rows.

Split the chosen body into system material, tool definitions, existing messages,
and other envelope bytes. Record both the raw byte length and the serialized
character count. If the provider offers an exact token-count endpoint and the
prompt is safe to send, keep that result in a separate column. Estimated and
provider-counted tokens should never silently share one label.

Now capture several turns in the same session. Use
`request_size(t) = fixed_prefix + growing_history(t)`, then plot serialized size
against turn number to expose the standing system-and-tool load at the
intercept. Conversation growth, tool-result accumulation, and repeated artifacts
determine the slope. Equal tenth-turn sizes can hide opposite diagnoses: one
agent started heavy; the other accumulated waste quickly.

Repeat the first-turn capture after disabling one server at a time. Diff tool
names and schema characters, then inspect actual invocation logs over a
representative week. A large server used daily may earn its place. One untouched
during the sampled week becomes a candidate for deferred loading or removal,
pending checks for seasonal and emergency use. Keep this as an inventory
decision, since a smaller schema can still grant dangerous access.

## Defer definitions without destabilizing the prefix

The blunt fix is to disconnect tools. That works for dead integrations. A broad
agent still needs a large catalog, and progressive discovery gives it access
without placing every full interface in the initial model context.

MCP's client guidance describes a catalog, inspect, execute pattern. The host
fetches definitions through `tools/list`, keeps them outside model context, and
offers a small search interface. Search returns names and short descriptions.
The model inspects a promising match to receive its full schema, then calls it.
Server-level discovery applies the same idea earlier by connecting only the
servers relevant to the current skill or task.

Provider-native tool search can handle the retrieval step. Anthropic's version
keeps a few frequent tools loaded and marks the rest with `defer_loading`. The
API accepts the complete catalog server-side, searches names and argument
descriptions, and expands selected references for the model. The stable prefix
stays intact as tools are discovered. A custom MCP host can use keyword search,
embeddings, or a permission-aware index. Search quality should be measured with
the same tasks the agent actually performs; attractive catalog compression is
useless when the needed tool disappears from the candidate set.

Caching needs two separate labels. Host-side caching of `tools/list` avoids a
network round trip to the MCP server. Provider prompt caching avoids reprocessing
a stable model prefix at the cold rate. Because adding or removing definitions
during a conversation can invalidate that cache, refresh on `list_changed`
without reordering the model's tool array every turn. Append discoveries after
the cache breakpoint, use a stable `call_tool({name, args})` interface, or make
server changes at a session boundary.

Programmatic tool calling attacks the other half of the graph: the slope. A
model writes a small script against typed tool stubs; a sandbox executes the
chain, filters intermediate data, and returns the final result. Thousands of log
records can move between brokered tools without entering model history. This
requires a real sandbox with network access denied, credentials held by the
host, and authorization checked on each brokered call. Approving a script must
never become blanket approval for every action it attempts.

The clean audit ends with four numbers per workload: first-turn schema
characters, fixed-prefix tokens as counted by the provider, cache-read share,
and history growth per turn. Re-run the task suite after every catalog change.
A smaller first request is useful evidence. Stable task completion and correct
tool selection decide whether the change was an improvement.

## Sources

- Local first-request capture, 7 August 2026: one machine, two trials, Claude
  real configuration; serialized characters measured exactly, tokens estimated
  at characters divided by 6.4, and no provider calls.
- [Anthropic, Tool search tool](https://platform.claude.com/docs/en/agents-and-tools/tool-use/tool-search-tool)
- [Model Context Protocol, Client Best Practices](https://modelcontextprotocol.io/docs/develop/clients/client-best-practices)
- [Harness-Bench: Measuring Harness Effects across Models in Realistic Agent Workflows](https://arxiv.org/abs/2605.27922)