Forty Tools in One Prompt: What It Actually Costs, and the Three Signals That Say Split the Agent

An agent that starts with six tools and grows to forty rarely feels like it broke. It feels like it got vague. It picks the almost-right tool, asks a clarifying question it already had the answer to, and occasionally calls a read endpoint when the task plainly needed a write. Nobody changed the system prompt, so nobody suspects the tool list.

The tool list is the change. Every tool you register is permanent prompt weight — a name, a description, a JSON schema, and an implicit invitation to consider it on every single turn. That cost is not just tokens. It is decision surface.

💰 The three costs, separated

Token cost is the one people notice and the least interesting. A tool definition with a handful of parameters and a decent description runs, say, 150 to 400 tokens. Forty of those is a fixed 6,000–16,000-token preamble on every request in the loop. Illustrative, but the shape is right: it is a tax you pay per turn, not per task, so a twelve-turn task pays it twelve times. Prompt caching softens it considerably and is why this cost stops being the binding one.

Selection cost is the real one. The model must discriminate between tools, and discrimination gets harder as candidates crowd together in description space. get_account, get_account_details, fetch_customer_record, and lookup_subscriber are four tools an ops platform grows honestly over two years. To the model they are near-synonyms. It will pick among them by surface cues in your wording, not by the distinctions living in your team's heads.

Instruction cost is the one that compounds. Every ambiguous tool pair eventually earns a line of prompt to disambiguate it: "use fetch_customer_record only when you need billing state; prefer get_account otherwise." Twenty of those lines become a rulebook the model must hold alongside the actual task. The rulebook competes with the user's request for attention, and the request is shorter.

Why "the model will figure it out" stops being true

Tool selection is a retrieval problem dressed as a reasoning problem. The model is matching the task against descriptions, and the failure mode is nearest-neighbour confusion, not stupidity. Two consequences follow.

First, accuracy degrades unevenly. Your ten distinctive tools stay reliable at any list size. Your four overlapping ones get worse as the list grows, because each new tool adds another chance to land in the wrong neighbourhood. Aggregate tool-accuracy metrics hide this completely — the average looks fine while one cluster rots.

Second, adding instructions to fix it works for a while and then inverts. Past some point the disambiguation prose is itself the context problem, and each new clarification slightly degrades adherence to the older ones.

🔍 The three signals that say split

Do not split on tool count. Count is a proxy and a bad one — a coherent thirty-tool database agent can outperform a twelve-tool agent whose tools span three unrelated domains. Split on these.

Signal 1: the confusion matrix clusters. Log every tool call with the task that prompted it, then build a matrix of intended tool against called tool. You are looking for off-diagonal mass that is concentrated. Scattered single errors mean bad descriptions; a dense block where four tools mutually confuse each other means those four belong behind one boundary, exposed as a single interface or handed to their own agent.

Signal 2: the instruction budget crosses the task budget. Count the lines of your system prompt that exist purely to tell the model which tool to prefer. When that section is longer than a typical user request, the agent is spending more of its attention on navigating itself than on the work.

Signal 3: no single session uses more than a slice. Pull a week of traces and, per session, record which tools were actually called. If the median session touches four tools and the sessions cluster into disjoint sets — billing sessions never touch the deployment tools, deployment sessions never touch billing — the sets are your split lines. They were already there; you were paying to keep them merged.

What splitting actually means

Splitting the agent is the heaviest of four options. Try them in order.

1. Merge and parameterize. Four near-synonym read tools become one get_account(view: "billing" | "usage" | "identity"). This is usually the biggest single win and it is not an architecture change. Fewer tools, sharper boundaries, the distinction moved from prose into a typed enum where it belongs.
2. Rewrite descriptions for contrast, not completeness. A description's job is to distinguish this tool from its nearest neighbour, so name the neighbour: "for the current balance only; use get_invoice_history for past charges." One sentence of contrast beats a paragraph of specification.
3. Gate tools by phase. Expose the read tools during investigation and swap in the write tools only after a plan is confirmed. Same agent, smaller decision surface per turn, and a real safety benefit as a side effect.
4. Split into sub-agents. A supervisor holds a small number of coarse capabilities and delegates to specialists, each with its own focused tool list. This buys clean decision surfaces and costs you a handoff — serialized context, a new failure mode where the supervisor mis-routes instead of mis-picking, and extra latency per delegation.

⚠️ Step 4 is not free. You have traded one confusion for another: routing errors at the supervisor are harder to debug than tool-selection errors inside one agent, because the specialist confidently does the wrong correct thing and returns a clean-looking result. Only pay that price when steps 1 to 3 have genuinely run out, and split along the disjoint-session boundary that signal 3 handed you rather than along your org chart.

The takeaway

Tool lists grow by addition and are almost never audited by subtraction. Put a standing item on the calendar: pull the confusion matrix, count the disambiguation lines, check whether sessions have separated into clusters. Most bloated agents do not need splitting — they need eight tools merged into three and a dozen descriptions rewritten to say what they are not. Reach for the second agent when the boundary is already visible in your traces, not when the list simply looks long.