The Triage Agent That Should Have Been a Filter

A support team put an agent in front of their ticket queue to route incoming requests. It worked. It was also the wrong tool for roughly four out of five tickets it handled, and the reason is worth generalizing.

Nothing about the deployment was incompetent. The agent read each ticket, classified it, checked the customer record, and assigned a queue. Accuracy was good. The team's complaint was that routing had become slower and more expensive than the rules it replaced, and that occasionally a ticket went somewhere baffling.

Where the work actually was

Look at what a real support queue contains. A large share of tickets are structurally identical to each other: password resets phrased six ways, "where is my order" with a tracking number attached, billing questions that always mention an invoice ID. These match on keywords, on the form they arrived through, or on a regex over an ID pattern. A deterministic rule routes them correctly every time, in microseconds, for free.

The interesting tickets are the residual — the ones that mention two problems, or describe a symptom without naming the product, or are angry in a way that should escalate regardless of topic. These are genuinely hard, they need judgment, and no rule catches them.

The deployment sent everything through the agent. So the majority of the volume — the part a rule handles perfectly — acquired a model call's latency, a model call's cost, and a model call's variance. The variance is the part that stings: a rule that routes password resets is correct 100% of the time, and an excellent agent doing the same job is correct almost that often. Replacing a deterministic step with an excellent probabilistic one is a downgrade whenever the deterministic step was already right.

The shape of the fix

Rules first, agent for what's left:

route(ticket):
    if match = deterministic_rules(ticket):     # keyword, form_id, ID pattern
        return match                             # ~majority of volume, instant
    return agent_route(ticket)                   # the residual, judgment required

The agent now sees a smaller, harder, more interesting stream — which also makes it better, because its context isn't diluted by trivia and its prompt can be written for genuinely ambiguous cases instead of trying to cover everything.

Two details make this work in practice:

The agent needs an escape hatch upward, not just a classification. For the residual, "I am not confident" is a legitimate and useful output. Route those to a human queue explicitly rather than forcing a guess. A confidently wrong routing decision costs more than an admitted uncertain one.

Rules must be allowed to grow. When the agent routes a class of ticket the same way fifty times, that's a rule waiting to be written. Feeding that back — periodically reviewing the agent's most repetitive decisions and promoting them into deterministic rules — keeps the expensive path shrinking over time instead of growing.

🔍 Finding your own residual

This generalizes past ticket routing to anything an agent classifies, extracts, or dispatches. The diagnostic is the same:

1. Log every agent decision with its input and output for a week. No changes to behavior, just the log.
2. Ask what fraction of those decisions a competent engineer could reproduce with rules — string matching, a lookup table, a regex, a database join. Be honest; the answer is often uncomfortable.
3. That fraction is what you're overpaying for. It is also, usually, the part with the tightest latency requirements.
4. Write rules for the top few patterns by volume, not for everything. The distribution is typically steep — a handful of patterns covers most of the easy traffic.

❌ The failure mode in the other direction

The opposite mistake is real too, and this argument shouldn't be read as an endorsement of it. Teams that respond by building a hundred-rule decision tree end up with something nobody can modify safely, that fails opaquely on anything unanticipated, and that requires an engineer to change every time the business does.

The residual is exactly where rules stop being cheap. The signal is a rule that needs a rule for its exceptions, or a classification that turns on tone, intent, or context rather than on tokens present in the text. When you find yourself writing those, you've found the agent's actual job.

The takeaway

An agent's advantage is judgment under ambiguity, and it charges for that advantage on every single call — including calls where there was no ambiguity to resolve. Put the deterministic path first, give the agent the residual, and let it hand back what it isn't sure about. The result is faster and cheaper on the bulk of traffic, and better on the part that was always hard.