The Expense Agent Logged "Rate limit exceeded" as a Vendor Name. Make Tool Errors Impossible to Mistake for Results.

A personal expense agent reads receipts from a shared folder, calls a lookup tool to resolve each merchant, and writes a row to a spreadsheet. One month the spreadsheet showed a vendor called "Rate limit exceeded, retry after 30s" with a category of "Software" and an amount of $42.18. Nothing crashed, and nothing warned anyone. The agent had done exactly what it was built to do with what the tool handed back.

The tool did not lie. It just returned its failure in the same shape as its success, and a language model has no reliable way to tell the two apart.

How a failure turns into a fact

Here is the merchant lookup as it was originally written:

def lookup_merchant(raw_name: str) -> str:
    try:
        resp = merchants_api.search(raw_name)
        return resp.best_match.display_name
    except RateLimitError as e:
        return str(e)
    except Exception:
        return "unknown"

From the model's side, every call returns a short string. "Blue Bottle Coffee" is a string. "unknown" is a string. "Rate limit exceeded, retry after 30s" is also a string. The model was asked for a vendor name, received a string in the slot where vendor names go, and moved on to categorising it.

Three properties of the agent loop make this worse than it looks:

• The model is primed to accept. The prompt said "use the lookup tool to get the vendor name." The tool result arrives framed as the answer to that request.
• Error text is plausible text. "Unknown", "N/A", "Not found" and "Timeout" all look like values a real field could hold. Some of them are real merchant names somewhere.
• The next step launders it. Once the error is written into the vendor column, the categoriser sees a vendor, not an error. By the time a human looks, the provenance is gone.

Prompting the model to "check whether the tool failed" helps a little and fails often. You are asking it to infer a status flag from prose, every time, under a prompt that rewards finishing.

🔍 The fix: give failure its own shape

The tool should return a structure where success and failure cannot be confused, because they don't share fields.

def lookup_merchant(raw_name: str) -> dict:
    try:
        resp = merchants_api.search(raw_name)
    except RateLimitError as e:
        return {
            "ok": False,
            "error_type": "rate_limited",
            "retryable": True,
            "retry_after_s": e.retry_after,
            "message": "Merchant service is throttling requests.",
        }
    except Exception:
        return {
            "ok": False,
            "error_type": "lookup_failed",
            "retryable": False,
            "message": "Merchant service call failed.",
        }

    if resp.best_match is None:
        return {"ok": True, "found": False, "vendor": None}

    return {"ok": True, "found": True, "vendor": resp.best_match.display_name}

Four things changed, and each one closes a specific gap.

1. An explicit ok flag. The model no longer has to guess. A boolean at the top of the result is the cheapest signal you can give it.
2. No error text in the value field. On failure, there is no vendor key at all. There is nothing to copy into the spreadsheet.
3. "Not found" is a success, not a failure. The service worked; it just had no match. Separating found: False from ok: False stops the agent from retrying a lookup that will never succeed, and stops it from treating an outage as "no such merchant".
4. A machine-readable reason with a retry hint. error_type and retryable let the agent decide what to do next instead of improvising from a sentence.

Back it up outside the model

The structured return makes the right behaviour easy. It does not make the wrong behaviour impossible. The model can still, occasionally, write "lookup_failed" into a cell. Two cheap checks in your own code close that hole.

Validate at the write tool. The spreadsheet writer is also a tool, so it can refuse bad input:

def write_expense_row(vendor: str, amount: float, category: str) -> dict:
    if not vendor or vendor.lower() in KNOWN_ERROR_TOKENS:
        return {"ok": False, "error_type": "invalid_vendor",
                "message": "Vendor is empty or looks like an error value."}
    ...

Short-circuit retryable errors in the harness. If a tool returns retryable: True with a retry_after_s, the loop can sleep and retry before the model ever sees the result. Only after the retries run out does the failure reach the model, and by then it's a genuine dead end the agent has to plan around.

✅ A checklist for any tool you hand to an agent

• ✅ Success and failure return different shapes, distinguished by a top-level flag.
• ✅ Error messages never appear in a field that holds real data.
• ✅ "Worked but found nothing" is distinct from "did not work".
• ✅ Each failure carries a short, stable error_type the agent can branch on.
• ✅ Retryable failures say so, with a delay if the upstream gave one.
• ✅ Tools that write data validate their inputs and refuse obvious garbage.
• ❌ No bare return str(e).
• ❌ No sentinel strings like "unknown" or "N/A" standing in for null.

Run that list over every tool in the agent, not just the one that bit you. The receipt-reader, the currency converter and the calendar tool all had their own return str(e) buried somewhere.

Where the responsibility sits

It's tempting to treat this as a model-quality problem: a smarter model would have noticed the vendor name was an error message. Sometimes it would. But a tool that returns failures as plausible values is pushing a classification job onto the model that the tool already had the answer to. The tool knew it failed. It caught the exception. It then threw that knowledge away by flattening it into a string.

Tool design is where you decide what the model has to guess and what it gets told. Every bit of state that the tool knows and the model has to infer is a place where the agent will sometimes infer wrong, and a mistake from one call gets copied into everything downstream of it. Make failure loud in the return shape, check it again at the point of writing, and the vendor column goes back to holding vendors.