One Search Tool or Five? Getting Tool Granularity Right

Two agents, same job. One has search_by_customer, search_by_date, search_by_status, search_by_amount, and search_by_product. The other has a single search_orders with five optional filters. Both are defensible designs, and each fails in a way the other doesn't.

Granularity is one of the few tool-design decisions that measurably changes accuracy, and it has a rule that resolves most cases.

What each design costs

Many narrow tools. Selection accuracy degrades as the list grows — more options, more overlap, more chances to pick a neighbouring tool that almost fits. Every tool also spends context on its spec, so twenty tools can consume more of the window than the task. The characteristic failure is calling search_by_date when the request was about a customer and a date range, because no single tool covered both.

One broad tool. Selection is trivial — there's one candidate. The difficulty moves into the arguments, and now the model must infer which combination of optional filters expresses the request. The characteristic failure is a call with a filter left off, or a filter set to something the schema technically permits but the backend can't satisfy, producing empty results the model then interprets as "no matching orders."

Neither is safer in general. They relocate the error.

The rule

Split when the tools require different decisions. Merge when they differ only in values.

search_by_date and search_by_customer are the same decision — search orders — differing in which filter is populated. → Merge. The model's job becomes filling parameters, which it does well when the schema is explicit.

search_orders and refund_order are different decisions with different consequences, different permissions, and different mistakes. → Keep separate, no matter how tempting a unified order_action(action=...) looks. A single tool with an action enum that spans read and write is the worst version of this: it makes a destructive call one enum value away from a harmless one, and it defeats per-tool permissioning entirely.

The test: if two candidate tools would ever appear in the same sentence of your requirements as alternatives to each other ("search by date or by customer"), they're one tool. If they'd appear in sequence ("search, then refund"), they're two.

Making the merged tool work

A merged tool succeeds or fails on its schema:

search_orders(
  customer_id?: string,
  status?: enum[pending, shipped, delivered, cancelled],
  placed_after?: date,       # ISO 8601
  placed_before?: date,
  min_amount?: number,
  limit: integer = 20
)
Returns up to `limit` orders, newest first, with a `total_matched` count
so you can tell a truncated result from a complete one.

Three things that schema does: enums remove invented status values; explicit formats remove date ambiguity; total_matched prevents the most common downstream error, where a truncated result gets treated as the full set.

✅ Also worth stating in the description: what happens with no filters. Either it's an error or it returns the most recent N — but say which, or the model will find out by trying.

When the list is genuinely long

Some domains need many tools. Two techniques help before granularity does:

• Filter the tool list per turn. Offer only the tools relevant to the current phase. An agent in an investigation phase doesn't need the write tools in context, and removing them improves selection and reduces blast radius.
• Group behind a router tool. One query_data(source, ...) where the source enum picks the backend, rather than fifteen sibling tools. Applies the merge rule at a larger scale.

⚠️ The version to avoid: keeping twenty tools and compensating with a long system prompt that explains when to use each. That's tool descriptions in the wrong place, and it's paid for on every single turn.

🔍 Diagnosing which problem you have

Run your eval set and classify each wrong call:

• Right tool, wrong arguments → your schema is too loose. Add enums, formats, and required fields.
• Wrong tool, arguments that would've been right → your list is too fine-grained or your descriptions overlap. Merge, or sharpen the boundaries.
• Right tool, right arguments, wrong interpretation of the result → the return shape needs documenting, not the input.

These three point at three different fixes, and treating them as one undifferentiated "accuracy" problem is why tool surfaces get iterated on for weeks without improving.

The takeaway

Granularity is a choice about where errors happen: in selection or in arguments. Argument errors are cheaper — they're constrainable by schema, catchable by validation, and fixable without touching anything else. So merge tools that share a decision, split tools that don't, and put the saved complexity into a schema tight enough that the wrong call can't be expressed.