Reading Your First Agent Trace: Wrong Tool, Wrong Argument, or Wrong Plan?

An agent did something unhelpful and you have the trace open. Most people read it start to finish, feel vaguely dissatisfied, and go edit the system prompt. There's a faster method, and it takes about five minutes.

Every agent failure sorts into one of four buckets, each with a different fix. Identifying the bucket is nearly all of the work.

Step 1 — Find the first wrong turn, not the wrong output

Scroll to where the agent produced its bad result, then work backwards to the earliest turn that was already wrong. Agents are cumulative: one bad step contaminates everything after it, so the last few turns are usually consequences rather than causes.

The first wrong turn is the only one worth analyzing. Mark it and ignore everything downstream.

Step 2 — Classify it

At that turn, ask three questions in order.

Was the information it needed present in the context?

Read the array as sent — the system prompt, the accumulated turns, the tool results — and look for the fact the agent should have used. If it isn't there, stop: this is a context failure, and no amount of prompting fixes it. Something was trimmed, summarized away, never retrieved, or never included. → Fix the assembly, not the model.

This is the most common bucket and the most often misdiagnosed, because the missing fact is obvious to you — you know it, so you assume it was there.

Did it pick the right tool?

If the information was present but the agent called the wrong tool, it's a selection failure. The cause is almost always tool descriptions that don't separate territories, or too many similar tools. Fix the descriptions; the system prompt is the wrong place for it.

A useful sub-check: was the right tool even offered on that turn? Dynamic tool lists mean the answer is sometimes no, which turns this into a plumbing bug rather than a model one.

Were the arguments right?

Right tool, wrong arguments is an argument failure, and it splits again:

• The correct value was in context but wasn't used → usually a loose schema. Add enums, formats, required fields.
• The correct value wasn't in context → it's a context failure wearing a different hat. Go back to bucket one.

If all three were fine and the step was still wrong, you have a reasoning failure — the model had what it needed, chose an available tool, filled it correctly, and the choice was still bad for the task. This is the rarest bucket, and the only one where changing the prompt or the model is the right response.

Why the order matters

Most people start at reasoning, because that's the interesting hypothesis. Reasoning failures are maybe the smallest category in a badly-behaved agent, and every hour spent rewriting prompts to fix a context failure is wasted — the new prompt is just as unable to use a fact that isn't there.

Working the buckets in order costs minutes and stops you from tuning around a plumbing bug.

💡 The one thing to print

If your trace viewer shows a rendered conversation rather than the raw request, get the raw request. You want the actual array sent to the model on that turn — every message, in order, exactly as the API received it.

Rendered views hide precisely what you need: what got trimmed, how a tool result was truncated, what the system prompt actually resolved to after templating. Investigations stall for hours on differences between the pretty view and the real payload.

Worked example

An agent was asked to summarize a support thread and include the customer's plan tier. The summary omitted it.

• First wrong turn: the summarization turn — the last one. Nothing downstream.
• Information present? Searching the array for the tier: the get_customer result is there, but it was truncated at 200 characters, and plan_tier sat past the cutoff.
• Verdict: context failure. The truncation limit on tool results was doing it.
• Fix: raise the limit for that tool, or better, have the handler return only the fields that matter so truncation never bites.

Prompt engineering would not have helped. "Always include the plan tier" would have produced an agent that includes a plan tier it can't see — which, in practice, means inventing one.

The takeaway

Four buckets, checked in order: context, selection, arguments, reasoning. Find the first wrong turn, read the raw array as the model received it, and answer the three questions. Most failures resolve before you reach the fourth bucket — and the discipline of checking in that order is what separates a five-minute diagnosis from an afternoon of prompt tweaking.