Extraction Was Never the Hard Part: An Invoice Agent, Rebuilt

The first version of the invoice agent read the PDF and typed the numbers into the accounting system. It worked most of the time, saved a bit of keying, and made the whole process less trustworthy than it had been before.

The problem was not accuracy in the abstract. It was that when the agent got a number wrong, nothing looked wrong. A total that should have been 1,450.00 went in as 145.000 because a vendor formatted decimals in a way the model smoothed over. Nobody noticed until the payment run.

What the work in accounts payable actually is

An invoice arrives, and three documents have to agree:

• The invoice — what the vendor says is owed.
• The purchase order — what was agreed in advance.
• The goods receipt — what actually showed up on the loading dock.

When the three agree, nothing interesting happens. The system matches them and posts the invoice with no human involved, and that part was already automated long before anyone considered an agent.

The work is the ones that disagree. A quantity is off by two. A price is higher than the PO. There is a line item nobody recognizes. Half the order arrived and the vendor billed for all of it. Those land in a queue, and a person opens three documents in three windows and works out which kind of disagreement it is.

Why version one aimed at the wrong half

Reading numbers off a document is a solved problem with unglamorous tools: OCR, layout templates, per-vendor parsers built up over years. They are not perfect. What matters is how they fail.

When a template does not match, the field comes back empty. That is a loud failure — an obvious gap that stops the process and routes to a person.

When a model reads a document, it fills the field in. Always. It produces a plausible number even when the layout confused it, and a plausible wrong number flows through to a payment.

⚠️ For any step where a wrong answer would be silent, prefer the tool that fails loudly. Version one replaced a system that said "I could not read this" with one that said "1,450.00" and meant nothing by it.

What version two did instead

The rebuild left extraction alone. The parsers keep doing the reading, and where they fail, they fail the way they always did.

The agent starts one step later. Its input is not a scanned document; it is a structured statement of a disagreement:

PO 44192   : 40 units @ 12.50 = 500.00
Invoice    :  4 cases @ 125.00 = 500.00
Receipt    : 40 units received, 2026-08-19
Mismatch   : quantity 40 vs 4, unit price 12.50 vs 125.00

And its job is a question a person can check: which of the known reasons explains this, and what should happen next?

The known reasons are a short written list, built from what the AP team already sees week to week:

• Unit-of-measure mismatch (cases versus units)
• Partial delivery
• Agreed price change or change order
• Freight or handling billed separately
• Duplicate invoice
• Tax treatment difference
• None of the above

That last one matters. A closed list makes the output a classification rather than an essay, and "none of the above" is a real answer that routes the case to a person with the comparison already laid out.

The worked example

For the mismatch above, the agent has tools to check vendor history and change orders. It finds that this vendor has billed by the case on every prior invoice, ten units to a case, and that no change order exists.

What it writes:

Unit-of-measure mismatch. This vendor bills by case at 10 units per case (last 6 invoices), so 4 cases = 40 units, matching the PO and the receipt. Effective unit price 12.50, consistent with PO 44192. No change order found. Recommend match and post.

A person reads two lines and approves, instead of opening three documents and reconstructing the same reasoning.

✅ What made the second version work

• The agent operates on structured facts, not scans. Everything it reasons about was extracted by something that fails loudly.
• Every claim cites its source. "Last 6 invoices" and "PO 44192" are checkable in seconds, which is what makes a fast approval safe rather than lazy.
• It recommends; a person disposes. Above a value threshold, or on "none of the above," a human decides.
• The reason list is written down. When a new kind of disagreement starts appearing, someone adds it deliberately, and that list becomes a decent record of how the vendor base actually behaves.

🔍 The question worth transferring

In any document-heavy workflow, two different jobs are hiding inside what looks like one:

• Reading — getting values off a page. Deterministic tools are good at this and fail visibly.
• Reconciling — deciding what it means when sources disagree. This needs context, history and judgement, and it is where people currently spend their afternoons.

Point the agent at the reconciling.

The exception worth naming: when there is only one document and nothing to cross-check — an unstructured contract, a single scanned form with no counterpart — reading is the job, and the design problem becomes making that reading loudly checkable instead of quietly plausible. Different post, different failure mode.

The invoice agent's value never showed up as fewer people. It showed up as the exception queue clearing in a morning instead of over three days, because the reasoning was already written down when someone opened each case.