Can You See the Raw Messages? The Question That Sorts Agent Frameworks

Every agent framework demos well. You register three tools, ask a question, and a tidy trace scrolls past. The differences only surface the first time an agent does something inexplicable in production and you need to know exactly what the model was looking at when it went wrong.

That moment has one prerequisite, and framework comparisons almost never lead with it: can you obtain the exact message array that was sent to the model on a given turn, and can you modify it before it goes?

Answer that and most of the rest of the evaluation follows.

Why this is the load-bearing question

An agent's behavior on any single turn is a function of one input: the messages plus the tool schemas, sent to a model at a temperature. Planning strategy, memory policy, context trimming, retry logic, guardrails — all of it eventually collapses into what bytes went over the wire.

A framework is, functionally, a machine for assembling that array. Everything else it offers is convenience on top. So when a run goes sideways, there are only two possible root causes: the array was wrong, or the model was wrong given a correct array. You cannot tell which without seeing the array.

Frameworks that hide it turn every incident into a guessing game about the framework's internals rather than an investigation into your agent.

🪜 Four levels of access

Frameworks land on a ladder. Where a framework sits matters more than its feature list.

Level 0 — Opaque. You pass a prompt and tools, you get a final answer. Intermediate steps are internal. Debugging means reading the library's source or adding print statements to someone else's code.

Level 1 — Rendered after the fact. There's a trace UI or a log stream, but what it shows is a view: prettified, truncated at some length, with tool results summarized and system content sometimes omitted. Close enough to follow the story, not close enough to reproduce it.

Level 2 — Exact and replayable. You can retrieve the literal request payload for turn 7, serialize it, and send it to the model API yourself outside the framework. Same input, same output. This is the level where real debugging starts.

Level 3 — Interceptable. A hook fires before each request with the assembled array, and whatever you return is what gets sent. You can drop a stale tool result, pin a system block, inject a correction, or redact a secret that leaked into context.

Level 2 is the floor for anything you intend to operate. Level 3 is what you want once your agent has an opinionated memory or context policy, because that policy is where most production behavior actually lives.

The ten-minute test before you commit

Don't read the docs for this. Run it.

# 1. Run any multi-step task.
run = agent.run("...")

# 2. Can you get the exact payload for one specific turn?
payload = framework.get_request(run.id, turn=4)

# 3. Is it the real thing, or a rendering of it?
#    Send it straight to the model API, bypassing the framework.
resp = client.messages.create(**payload)

# 4. Does it produce the same tool call the agent made at turn 4?
assert resp.content[0].name == run.steps[4].tool_name

Three checks hide inside that snippet:

• Fidelity. If step 3 raises a schema error, what you retrieved was a display object, not a request. That's Level 1 wearing a Level 2 badge.
• Completeness. Diff the payload against what you believe you configured. Injected system text, tool-description rewrites, and auto-inserted reminders show up here and nowhere else.
• Determinism. If the replay diverges wildly at temperature 0, something stateful lives between your call and the wire.

Then try to change it: add a hook, mutate one message, confirm the mutation reaches the model. If there is no such seam, you are at Level 2, permanently.

💸 What each missing rung costs

Consider a research agent that reads sources, extracts claims, and writes a brief. It starts producing briefs that quietly omit the most recent source.

At Level 3, this is twenty minutes: dump the payload for the writing turn, see that the newest tool result was evicted by a context-trimming rule that sorts by result size rather than recency, change the rule in the hook, done.

At Level 2, it's an afternoon. You find the same missing result, but the trimming lives inside the framework. Your fix is a fork, a config flag that may not exist, or a workaround that re-injects the source through a different channel.

At Level 1, you never find it. The trace shows the tool call succeeding and the brief being written. The eviction happened between those two rows, in the gap the rendering doesn't show. You will conclude the model is "being lazy" and spend a week on prompt wording.

That misdiagnosis is the real cost. Missing visibility doesn't just slow you down — it reliably routes you to the wrong fix, because the only layer you can see is the prompt, so the prompt is what you'll blame.

When a closed loop is genuinely fine

Not every agent needs Level 3, and the ladder is not a purity test.

• ✅ Prototypes and internal tools where a bad answer costs a shrug.
• ✅ Narrow, short workflows — two or three tools, four or five turns — where the whole context fits in your head anyway.
• ✅ A vendor-operated product where the vendor, not you, is on call. You're buying an outcome, not a runtime.

❌ The mismatch to avoid is picking a Level 0 or 1 framework for a long-horizon agent with memory, retrieval, and a dozen tools, then discovering at incident time that the component you most need to inspect is the one component you don't own.

The part nobody budgets for

Agent frameworks churn fast. APIs shift, abstractions get rewritten, projects get absorbed or abandoned. The message array is the one artifact that outlives all of that, because it's just the model provider's request format.

If your evals, your golden files, and your regression checks are all expressed as message arrays in, tool calls out, the framework becomes a swappable convenience layer. Swap it and your test suite still means something. If those same assets are expressed in framework-specific run objects, migrating means rewriting your entire safety net at exactly the moment you're least confident.

Pick the framework whose internals you can hold in your hand. Convenience you can add later. Visibility you can only choose up front.