When to Stop Improving an Agent

Agent improvement has no natural stopping point. There's always another failing case, another prompt refinement, another routing decision. Without a stopping rule, teams either optimize indefinitely past the point of return or abandon the work arbitrarily when attention moves elsewhere.

Both are avoidable with an explicit answer to "what would good enough look like?"

Define good enough before you start

The number should come from the decision the agent serves, not from a general aspiration:

This agent replaces a manual process with an 8% human error rate.
Target: ≤5% error, with errors detectable within one day.
Below that: continue. At that: stop and operate.

⚠️ Without this, the implicit target is "as good as possible," which is unbounded and guarantees the work never ends.

Note the second clause. Detectability often matters more than the rate. An agent with a 5% error rate where errors are caught quickly is more usable than one at 2% where they're silent — and improving detectability is frequently cheaper than improving accuracy.

The signs you've hit diminishing returns

Effort per point of improvement is rising sharply. Early fixes are structural — a tool description, a missing check, a context problem — and each moves the number meaningfully. Later ones are case-specific and move it slightly.

Remaining failures are the genuinely hard tail. Ambiguous, unusual, or requiring information that doesn't exist. These don't yield to engineering, and the right response is routing rather than solving.

Fixes start trading off against each other. Improving one case degrades another. That's the signature of having reached the limit of what the current design expresses, and further tuning is moving the problem around.

The prompt is getting longer with each fix. Accumulating patches rather than addressing structure — and adherence to any individual instruction falls as the document grows.

✅ What to do instead of continuing

Route the residual. Detect the hard cases and send them to a human. This converts an unsolvable accuracy problem into a solvable detection problem, which is usually much more tractable.

Improve detectability rather than accuracy. If wrong answers get caught fast, the error rate matters less.

Shrink the scope. An agent doing three things adequately is often better replaced by one doing one thing well plus a clear boundary.

Fix the input. Frequently the residual failures trace to unclear specifications, missing documentation, or bad data. That work helps everything, not just the agent — and it's the highest-return option that isn't about the agent at all.

Stop and operate. Move to monitoring, sampled review, and maintenance. The work shifts from improvement to keeping it from degrading, which is a real job with a real cost.

💡 The re-evaluation trigger

Stopping isn't permanent. Write down what would justify resuming:

• A model release crossing a threshold on your eval set.
• The failure distribution changing — new traffic patterns, new failure categories.
• A change in what the residual costs, if volume grows.
• A new oracle becoming available, which changes what's measurable.

Then check on a cadence rather than continuously. This is what turns "we stopped" into "we stopped for reasons, and here's what would change it."

🔍 The number that says stop

Track effort per point of improvement across your last several improvement cycles. When it rises steeply and the remaining failures are the hard tail, you're done.

That's a more honest signal than the absolute error rate, because it measures whether continued work is producing anything — and it's visible in your own history rather than requiring a judgment call.

The takeaway

Set the target from the decision the agent serves, and include detectability alongside accuracy — improving how fast errors surface is often cheaper than improving how often they occur. Stop when effort per point of improvement rises sharply and the residual is the genuinely hard tail. Then route the residual, fix the inputs, and move to operating it — with written triggers for when it's worth reopening.