The Code-Review Agent That Commented on Everything

A code-review agent was added to a repository's pull requests. Within three weeks the team had muted it. It wasn't wrong — most comments were technically defensible — but a twelve-line PR would attract nine comments, and reviewers began scrolling past the whole block to find the human feedback underneath.

The agent was doing what it was asked. The task was framed as "review this diff and comment on issues," and there is no diff without issues if you look hard enough.

Why review agents over-comment

The task rewards output. An agent asked to find problems that returns nothing feels like a failure — of the agent and, implicitly, of the person who deployed it. Nothing in the framing makes silence a good outcome.

Every codebase violates some defensible principle. Naming, error handling, test coverage, a magic number, a function slightly too long. All real, most not worth a PR comment on a two-line change.

There's no cost signal. A human reviewer feels the cost of a nitpick — the author's time, the relationship, their own credibility. An agent's tenth comment costs it nothing, so nothing dampens it.

Severity gets flattened. A missing null check and an inconsistent variable name arrive as two comments in the same visual weight, and the reader has to sort them.

Give it a budget and a bar

The changes that fixed it were structural, not prompt tweaks.

Cap the comments. Three per PR, hard limit in code. This forces ranking: the agent must decide which three matter, which is a much better task than "list everything." → The quality of the top three is dramatically higher than the average of nine, because ranking is a capability the model has and was never asked to use.

Scale the cap to the diff. One comment for a small PR, up to five for a large one. A twelve-line change attracting nine comments was the specific thing that broke trust.

Require a severity, with a defined bar. Only blocking and should_fix get posted; nitpick goes into a collapsed summary or nowhere. Making the agent classify forces the distinction it was previously eliding.

Make "no comments" an explicit, successful output. A approve_with_no_comments(reason) tool, and eval cases where calling it is the correct answer. Without those cases, the agent never learns silence is available.

Point it at what humans are bad at

The second insight was about placement. The agent was reviewing the same things a human reviewer would notice — naming, structure, style — which is where humans are strong and tooling already exists.

Redirecting it to what humans reliably miss made it valuable:

• Cross-file consistency. A caller not updated when a signature changed, three files away from the diff.
• Invariants stated elsewhere. A comment in another module saying "callers must hold the lock," violated here.
• Historical context. "This line was changed in the fix for INC-4471; the change reverts it."
• Missing test coverage for the specific branch introduced, rather than coverage in general.
• Error paths — the code path nobody reads, where a resource isn't released or an error is swallowed.

Each of these requires reading beyond the diff, which is exactly what a human reviewer is least likely to do and an agent can do cheaply. ✅ A comment that says "this changes behavior that billing/reconcile.py:88 depends on" is worth ten style notes.

What to measure

Comment count is the wrong metric in both directions. Better:

• Acted-on rate. What fraction of comments produce a change or a reply. Below a threshold, the agent is noise and the cap should tighten.
• Comments per hundred lines. Watch this over time; it drifts upward as prompts accumulate instructions.
• Human-found issues the agent missed, sampled from post-merge bugs. This is the real measure of value, and it's the one that justifies keeping it.
• ⚠️ Mute rate. If reviewers are collapsing or ignoring the block, nothing else matters.

The generalizable lesson

Any agent whose job is to find things will find them, because finding nothing looks like failing. Reviewers, auditors, linters, monitors, critics — all share this shape.

The fix is the same each time: cap the output, force ranking within the cap, require a severity classification, make "nothing to report" a valid and tested outcome, and point the agent at what the existing humans and tools miss rather than at what they already cover.

The takeaway

A review agent that comments on everything gets muted, and a muted agent has zero value regardless of how correct it was. Give it a small budget so it must rank, a bar so trivia doesn't qualify, permission to say nothing, and a mandate covering what human reviewers actually miss. Three good comments beat nine defensible ones, and it's the ranking — not the finding — that makes them good.