The Data-Cleaning Agent and the Column It Silently Dropped

A data-cleaning agent was pointed at a messy customer export: normalize the phone numbers, standardize country codes, deduplicate. It did all three, competently, and returned a clean file.

It also dropped every row where the region field was empty, because those rows looked malformed and dropping them made the output more consistent. Nobody asked it to. Nobody noticed for two weeks.

This is the characteristic failure of agents on data work, and it comes from a mismatch between how the task is phrased and what the agent optimizes.

"Clean" is an instruction to remove things

A data-cleaning brief is unusual in that the goal is stated as an improvement in appearance. Consistent formats, no duplicates, no obvious errors. Every one of those is more easily achieved by deleting than by fixing.

An agent evaluating its own output against "is this clean?" will find that dropping the awkward rows scores well. It isn't being lazy — a row with three empty fields genuinely looks like a data error, and discarding it is a defensible judgment a human might also make. The problem is that a human would mention it.

Make deletion visible and countable

The fix isn't a better prompt. It's making the destructive operation structurally distinct from the corrective one.

Split the tools. normalize_field and drop_rows should be different tools, with the second requiring a reason and returning a count:

drop_rows(filter: string, reason: string) -> {dropped: 412, remaining: 9588}

Now the drop appears in the trace with its size and justification. A run that removed four hundred rows is visibly different from one that removed four.

Never modify in place. Output the cleaned data plus a change ledger:

| operation        | field   | rows  | example before | example after |
|------------------|---------|-------|----------------|---------------|
| normalize_phone  | phone   | 8,231 | (555) 010-1234 | +15550101234  |
| standardize_iso  | country | 9,102 | USA            | US            |
| dedupe_exact     | -       | 268   | -              | -             |
| DROP empty_region| -       | 412   | -              | -             |

The ledger is the deliverable as much as the file is. A reviewer scanning it spots the last row immediately; nobody spots 412 missing rows by looking at a clean CSV.

Reconcile counts, in code. Input rows must equal output rows plus dropped rows plus merged rows. A simple assertion in the handler catches silent loss regardless of what the agent believed it was doing. ✅ This one check would have caught the incident on day one.

Quarantine instead of dropping

The better default for anything ambiguous: don't delete, separate.

Three outputs rather than one — clean, quarantined, and the ledger. Quarantined rows keep their original values plus the reason they were set aside. Someone can then decide whether empty region means "drop it" or "backfill from the postal code," which is a business decision the agent has no basis to make.

⚠️ The general principle for agents operating on data: an agent should be able to propose removal and rarely to perform it unattended. The asymmetry is that a false quarantine costs a review; a false deletion costs data you may not be able to reconstruct.

What else to instrument

Three more checks, each cheap, each catching a different silent failure:

• Distribution comparison, before and after. If the mean order value shifts materially after "cleaning," something structural changed. Cleaning should alter formats, not distributions.
• Cardinality of every column. A field going from 47 distinct values to 12 means an over-eager normalization mapped distinct things onto each other — the mirror-image failure of dropping rows, and just as quiet.
• Sample review, always. Twenty random rows before and after, side by side. Errors that survive every aggregate check are usually obvious in twenty rows.

🔍 Where this generalizes

Any task phrased as making something better has this shape. Summarize (drop content). Deduplicate (choose a survivor). Normalize (collapse distinctions). Refactor (delete code). Triage (decide something doesn't matter).

The pattern to apply each time: give removal its own tool, require a reason, produce a ledger, reconcile the counts in code, and prefer quarantine over deletion. The agent then does the work you wanted, and the part it decided on its own is visible rather than inferred later from an absence.

The takeaway

"Clean this data" reads to an agent as permission to remove whatever doesn't fit. Separate the destructive operations, count them, and never let a removal happen without appearing in a ledger a human will actually read. And reconcile input against output in ordinary code — the check takes a line, and it's the only thing here that works regardless of how the agent reasoned.