Agents Get Worse While You're Not Touching Them

Ordinary software sitting untouched in production keeps doing exactly what it did last year. An agent doesn't. Nobody changes anything, and six months later it handles a class of request worse than it used to.

That property — degradation with no change on your side — is the part of agent economics that budgets consistently miss. The build is estimated carefully; the recurring calibration cost that follows arrives unplanned, on a team that has moved to something else.

Five things that decay

The model moves. Providers deprecate versions and release new ones. Every migration is an evaluation project, not a config change: tool selection shifts, verbosity shifts, cost per task shifts. Stay on a floating alias and this happens to you silently and on someone else's schedule.

The eval set rots. A suite built from last spring's traffic slowly stops representing what users now send. Product changes, seasons change, a new customer segment arrives with different phrasing. The suite keeps passing, measuring a distribution that has moved.

Prompts accumulate. Each incident adds a line. Nobody removes any, because nobody can say which are load-bearing. Adherence to every individual instruction falls as the document grows, so the agent gets slowly worse at the things it was already doing.

Tools drift. Upstream APIs rename fields, change defaults, alter error semantics. Remote tool definitions can change without a deploy on your side at all — a description reworded by its maintainer is a prompt change you didn't make.

The domain changes. Refund policy updates, a product is renamed, the escalation path moves to a different team. The agent encodes assumptions that quietly expire, and it goes on applying them confidently.

The comparison people get wrong

The mental model most budgets use comes from conventional software:

traditional:   high build cost → low maintenance, mostly reactive
               (something breaks, you fix it)

agent:         moderate build cost → steady maintenance, mostly proactive
               (nothing breaks, and it degrades anyway)

→ The distinguishing property is that agent maintenance is not triggered by failure. There's no crash, no alert, no bug report — a slow drift in a system whose correctness was always statistical. Waiting for something to break means waiting for a threshold nobody set.

This is closer to how a forecasting model or a fraud rule set behaves than to how a web service behaves. Those disciplines already budget for periodic recalibration as a normal cost of operation, and agents belong in that category.

✅ What actually reduces the recurring cost

Not all of it is inevitable. Four choices at build time make a large difference to what you pay every quarter afterwards:

• Deterministic logic in code, not in the prompt. Code doesn't drift. Every rule you move out of the prompt is a rule that stops needing re-verification.
• Narrow scope. Maintenance cost scales with surface area — tools to re-verify, cases to re-check, behaviors that can shift. An agent that does one thing well is far cheaper to keep working than one that does six things adequately.
• Pinned versions everywhere — model, prompt, tool definitions. Pinning doesn't prevent change; it converts change from something that happens to you into a decision you make. That's the whole difference between a migration and an incident.
• Evals in the repository, growing from real failures. A suite that grows as production surprises you stays representative for free, rather than needing a re-sampling project.

🔍 Signs the maintenance isn't happening

Cheap to check, and each points at a specific accumulating debt:

• Nobody can say when the eval set was last refreshed against real traffic.
• The system prompt contains lines nobody can explain the origin of.
• The model is referenced by a floating alias rather than a pinned version.
• No single person owns a quality number for the agent.
• Nobody has read a sample of actual outputs in the last month.

That last one is the most telling. An agent whose outputs no human has looked at recently is being trusted on the strength of the fact that nothing has visibly broken — which, for this class of system, is not evidence.

What to tell whoever approves the budget

The honest framing is not "we'll build it and then maintain it," which invites the usual small maintenance allocation. It's:

This is a system with an ongoing calibration requirement. It needs a recurring share of engineering time — reviewing outputs, refreshing evaluation cases, migrating model versions, and pruning accumulated instructions — for as long as it runs. Without that, it degrades on its own.

Teams that say this up front get a sustainable allocation. Teams that don't end up with an agent nobody owns, quietly getting worse, until someone notices and proposes rebuilding it — which costs more than the maintenance would have.

The takeaway

Build cost is the smaller and more visible half. The recurring half — model migrations, eval refreshes, prompt pruning, tool drift, domain change — arrives whether or not anyone budgeted for it, and it isn't announced by anything breaking. Reduce it at design time by putting deterministic rules in code, keeping scope narrow, and pinning every version. Then fund the rest honestly, because an agent nobody is calibrating is an agent slowly becoming wrong.