One Prompt Per Job Beats One Prompt For All: The Router Pattern

A system prompt that has to cover six different kinds of request ends up serving none of them well. Each new case adds instructions; the instructions accumulate; adherence to any individual one drops. Everyone recognizes the two-thousand-word prompt with sections nobody dares delete.

The fix isn't better prompt writing. It's routing: classify first, then run a small prompt built for that class.

What the pattern is

Split into two stages.

Stage one classifies the request into one of a handful of known intents — cheap, fast, one call, no tools. Stage two runs the agent with the system prompt, tool list, and constraints specific to that intent.

intent = classify(request)          # small model, no tools, enum output
config = CONFIGS[intent]            # prompt + tools + limits + approval policy
return run_agent(request, config)

Each config is small enough to read in one sitting, and changing one cannot affect the others. That isolation is the whole point.

Why it works better than one big prompt

Instructions stay in scope. A refund-handling prompt containing only refund rules gets followed more reliably than the same rules buried among five other sections. The model isn't weighing them against unrelated guidance.

Tools shrink per route. A billing route doesn't need the deployment tools. Fewer tools improves selection accuracy and reduces blast radius simultaneously — the rare change that helps quality and safety together.

Changes stop being scary. With one prompt, every edit risks the behaviors you didn't test. With routes, the blast radius of an edit is one route, and your eval suite can be organized to match.

Different routes can have different everything — models, turn limits, approval requirements, temperature. A read-only lookup route can run cheap and ungated. A refund route can use a stronger model and require approval. Expressing that in a single prompt means asking the model to police its own risk tier, which it does inconsistently.

Where routers fail

The misroute. Classification is a model call, so it's wrong sometimes, and a misroute puts the request in front of an agent without the tools to serve it.

Mitigations: make "unclear" a valid class rather than forcing a guess; give each route an explicit escape (reroute(new_intent, reason)) so a run that discovers it's in the wrong place can say so rather than muddling through; log every reroute, since a route with a high reroute rate has a boundary problem.

The multi-intent request. "Cancel my order and update my address" spans two routes. Handle it deliberately — either decompose into two runs, or define a compound route. ⚠️ The version to avoid is a classifier that silently picks the first intent and drops the second, because the user gets a confident partial answer.

Route proliferation. Twenty routes is a maintenance problem of a different shape, with duplicated instructions drifting apart across configs. Keep genuinely shared rules — tone, safety, output format — in a shared preamble, and keep only the differences in each route. If two routes' configs are nearly identical, they're one route.

✅ When to reach for it

• Requests fall into a few naturally distinct kinds, and you can name them.
• Different kinds need different tools or different risk handling.
• Your system prompt has grown sections that don't apply to most requests.
• You want per-kind eval suites and per-kind metrics.

❌ When not to

• Requests are genuinely open-ended and don't cluster. A forced taxonomy over unclusterable input misroutes constantly.
• There are only two cases and they share most instructions — an if in the prompt assembly is simpler and honest.
• The classification is as hard as the task. If deciding the intent requires tool calls and investigation, routing hasn't saved anything.

🔍 Deriving the routes from data

Don't design the taxonomy in a meeting. Take a few hundred real requests and cluster them by what the agent needed to do — which tools, which permissions, which constraints — rather than by topic. Tool requirements make a better routing dimension than subject matter, because that's what actually differs in the config.

The residual that fits nowhere becomes the general route, and its size tells you whether routing suits your traffic at all. A general route absorbing most requests means the pattern isn't buying you much.

The takeaway

A prompt is a document, and documents lose coherence as they accumulate obligations. Routing keeps each one small, scoped, and independently testable, at the cost of one cheap classification call and the discipline to handle misroutes explicitly. If your system prompt has a table of contents, this is the refactor.