What Happens to QA When Agents Write the Tests

Test generation is one of the things agents do well, which puts pressure on a role partly defined by producing tests. The role doesn't disappear; what it's for changes, and the change is more favourable than it first appears.

What test generation actually covers

Agents are good at generating tests given a contract — a specification, a type signature, a documented behavior. Boundary enumeration, combinatorial coverage of flags, filling gaps in an existing suite: all genuinely faster.

What they're poor at is knowing what should be true. A test generated from an implementation asserts what the code does, which is exactly the trivial-assertion failure — coverage without verification.

→ Which locates the durable work precisely: deciding what should be true is the job; writing the assertion is not.

What the role becomes

Specifying correct behavior. Someone has to say what the system is supposed to do, including the cases nobody thought about. This was always the valuable half of testing and it was hidden inside the writing.

Designing the oracle. For anything without an obvious check, finding the mechanical verification. This is a genuinely technical contribution and it's the highest-leverage thing in the role.

Validating that tests test something. ✅ The mutation check — break the implementation, confirm the test fails. This becomes central rather than optional once tests are cheap to produce, because cheap production means more tests that verify nothing.

Exploratory testing. Using the system as a user would, looking for what nobody specified. This is human by construction: it's the search for cases outside the specification, and an agent works from the specification.

Risk judgment. Where does failure hurt, and where should the effort go? An agent will test uniformly; knowing where uniform is wrong is judgment.

Owning the eval set for agent-based features. The natural extension — the skills are the same, applied to a probabilistic system.

⚠️ The pressure that's real

Being honest: if a QA role is defined by writing test scripts to a specification, that's the exposed portion, and it's a large share of some jobs.

The transition is toward specification, oracle design, and risk judgment — which are the parts experienced testers were often already doing and rarely credited for. The people most exposed are those whose role was purely execution, and that's a real displacement rather than a reframing.

💡 The counterintuitive part

Cheap test generation makes test quality matter more, not less.

When writing tests was expensive, the suite was small and each test was deliberate. When generation is cheap, suites grow fast and fill with tests that pass and verify nothing — a suite that's large, green, and worthless, which is worse than a small one because it produces false confidence.

Someone has to be responsible for whether the suite actually catches things. That's a bigger job than it was, and it's the thing the role becomes.

✅ Practices for the transition

• Mutation testing as standard. Every generated test must fail against at least one broken version. This is the single highest-value check once generation is cheap.
• Generate from contracts, not implementations. Withhold the implementation so tests encode intent rather than behavior.
• Review the case list before the code. Twelve case descriptions take a minute to check; twelve test functions take much longer and get skimmed.
• Report mutation score alongside coverage. Coverage alone stops being meaningful.
• Keep exploratory time protected. It's the part that finds what nobody specified, and it's the first thing squeezed by throughput pressure.

The takeaway

Agents generate tests well from a contract and badly from an implementation, which locates the job: deciding what should be true, designing the oracle, verifying the tests verify something, exploring outside the specification, and judging where risk lives. Cheap generation makes suite quality a bigger responsibility, not a smaller one — because a large green suite that catches nothing is the default outcome without someone owning it.