The Migration Agent: 4,000 Files and What Made It Tractable

Large mechanical migrations — a framework upgrade, an API rename, a logging library swap — are the coding task agents suit best. The work is repetitive, the correctness criteria are objective, and no human enjoys any part of it.

They're also where an agent can produce four thousand plausible-looking diffs that nobody can review. The difference between the two outcomes is structural, and it's decided before any file is touched.

Establish the oracle first

Before the agent runs, there must be a mechanical answer to "is this file correctly migrated?" Usually a combination of: the project builds, the tests pass, no forbidden pattern remains, and the diff touches only expected constructs.

Without this you get four thousand diffs and a review queue nobody will finish. With it, most files are verified automatically and human attention goes to the residue.

A migration where you can't build this check isn't ready for an agent yet. → Building the checker is the migration's real first task, and it's usually a day of work that saves weeks.

One file, one run, one commit

The instinct is one agent processing everything. The version that works treats each file as an independent unit:

for file in files:
    run = agent.migrate(file, rules=RULES, examples=EXAMPLES)
    if verify(file):
        commit(file, message=f"migrate: {file}")
    else:
        quarantine(file, reason=verify.failure)

Four properties fall out of this. Failures are isolated — one weird file doesn't corrupt the batch. Each unit is reviewable as a small diff. The work is trivially parallel. And it's resumable, because completed files are committed.

⚠️ If files must be migrated in parallel and the tool edits a shared checkout, use isolated working copies. Two agents editing the same tree concurrently produces interleaved changes that fail confusingly and are hard to attribute.

Order matters: harvest examples early

Don't process files in arbitrary order. Start with a small, deliberately varied sample and have a human review those diffs closely.

That review produces two things: confirmation the rules are right, and a set of verified examples to include in the prompt for the remaining files. A migration agent with five real before/after pairs from this codebase is substantially more accurate than one working from an abstract description.

Then process the rest in order of increasing complexity — smallest and most typical first. By the time the agent reaches the awkward files, the pattern is well established in the examples, and you've already learned what the failure modes look like on cases that were cheap to check.

Quarantine liberally

Files that fail verification, or that the agent flags as unusual, go to a quarantine list rather than being retried indefinitely. Attempting a hard file three times produces three wrong diffs and a lot of tokens.

The quarantine list is the deliverable's second half. A migration that converts 92% automatically and hands back a precise list of the remaining 8% with reasons is a success. One that claims 100% and buries the questionable conversions in the bulk is worse than doing nothing, because now the review has to cover everything.

Track what the diffs contain, not just whether they pass

Verification passing isn't sufficient — a build can pass with a behavioral change. Two cheap checks add real safety:

Diff shape analysis. Cluster the diffs and look at the distinct shapes. A migration should produce a small number of recurring patterns. A file whose diff shape is unique is worth a human look regardless of whether it verified.

Line-count sanity. A file whose diff removes far more than it adds, in a migration meant to be roughly one-for-one, is a file where something was dropped. ✅ This catches the silent-deletion failure that every "make it consistent" task invites.

What a human still has to do

• Review the initial sample properly, because everything downstream inherits those decisions.
• Review every quarantined file.
• Review every unique diff shape.
• Decide on any case where the migration rule is genuinely ambiguous — the agent should surface these, not resolve them.
• Run the integration and end-to-end tests the per-file check can't cover.

That's a few hours across four thousand files, which is the actual value proposition. It isn't "no review" — it's review concentrated where judgment is needed.

The takeaway

Mechanical migrations suit agents when the work is decomposed into independently verifiable units. Build the checker before the agent, do a reviewed sample first and reuse those diffs as examples, process one file per run with its own commit, isolate concurrent work, and quarantine rather than retry. Then review the sample, the quarantine, and the unusual diff shapes — and let the verified bulk pass without reading four thousand files.