The Reliability Ceiling and What Would Have to Change

Agent deployments plateau at a reliability level below what teams want, and further prompt work stops helping. Understanding what sets that ceiling explains why effort stops paying, and what would actually move it.

The ceiling is compounded, not single

An agent run is a sequence of steps, each with an independent chance of going wrong. Say each step is right 98% of the time — high, for a step involving judgment.

A three-step task: roughly 94% of runs correct. A ten-step task: roughly 82%. A twenty-step task: roughly 67%.

⚠️ Those numbers are illustrative arithmetic, not measurements — real steps aren't independent and error rates vary enormously. But the shape is the point: per-step reliability that sounds excellent produces per-task reliability that isn't, and the gap widens with task length.

This is why a model improvement that raises per-step accuracy modestly can feel like it changed nothing on long tasks, and why the same agent seems fine in demos (short tasks) and disappointing in production (long ones).

What sets the per-step rate

Ambiguity in the step's inputs. A step that requires guessing has an error rate floor set by how often the guess is wrong, not by model quality.

Absence of a check. A step nobody verifies contributes its full error rate downstream. A step with a mechanical check contributes approximately zero, because failures get caught and retried.

Context quality. A step whose deciding information is missing or buried fails at a rate set by the context assembly, not the model.

Task-shape novelty. Steps following a familiar pattern are far more reliable than steps requiring an unusual approach.

What actually raises the ceiling

Given the compounding, the highest-leverage interventions aren't about making steps better — they're about changing the arithmetic.

Fewer steps. Batch tools that turn five calls into one remove four opportunities to fail. This is the most underrated reliability intervention available, and it improves cost and latency at the same time.

Checks between steps. Every step with a mechanical verification stops propagating errors. → A ten-step task with checks after steps three, six, and nine behaves like three short tasks rather than one long one, which is a completely different reliability profile.

Removing ambiguity at the source. A step that doesn't have to guess doesn't have a guessing error rate. Better specifications, explicit defaults, enumerated options.

Deterministic steps where possible. Anything moved from the model into code contributes zero error. The reliability argument for putting deterministic logic in code is stronger than the correctness argument.

Early failure rather than late. A run that stops at step four when something is wrong is better than one that completes twenty steps on a bad foundation — the second wastes more and is harder to diagnose.

✅ The ceiling that isn't about arithmetic

Some of the plateau isn't compounding at all:

The tail of genuinely hard cases. Every task distribution has inputs that are ambiguous, malformed, or unusual. No amount of engineering makes those reliable, and the right response is detecting and routing them rather than solving them.

Specification errors. A run that faithfully executes a wrong requirement is 100% reliable and wrong. This isn't visible in any per-step metric.

Unrecorded context. Steps depending on information that exists nowhere have an error floor set by how often the missing information mattered.

🔍 Finding your own ceiling's cause

Instrument per-step outcomes, then ask:

• Where do failures concentrate? A single step dominating means fix that step, not the model.
• How many steps does a typical task take? If the answer is large, reducing it is likely the fastest win available.
• Which steps have a mechanical check? The unchecked ones are contributing their full error rate.
• What fraction of failures are the hard tail? That fraction is your realistic ceiling until you route it to humans.

The takeaway

Per-task reliability is per-step reliability compounded, which is why excellent-sounding step accuracy produces disappointing task outcomes on long runs. Raise the ceiling by shortening the chain, inserting checks that stop propagation, moving deterministic work into code, and removing ambiguity at the source — all of which are engineering, not prompting. Then measure what fraction of remaining failures are the genuinely hard tail, because that's the number that tells you when to stop optimizing and start routing.