Let the Model Pick the Transition, Not the Tool

A returns agent has eleven tools. One of them is refund_order. In a free-form loop, that tool is on the menu from the first message onward, which means the agent can refund an order before it has verified who it is talking to, before it has checked whether the item is inside the return window, and before it has told the customer what the resolution will be.

The prompt says to check eligibility first. It says so twice. The agent mostly complies, and "mostly" is the entire problem — a prompt instruction is a request, while a tool schema is a fact about what exists.

The mismatch underneath

In a free-form agent the action space is constant: every tool, every turn. But the workflow's legal action space is not constant. It depends on what has been established so far. Refunding is illegal until eligibility is known; proposing a remedy is meaningless until an order is in scope.

Encoding that in prose asks the model to enforce a constraint. Encoding it in the tool list means the constraint holds whether or not the model was paying attention.

Step 1: derive states from preconditions, not activities

The instinct is to name states after what the agent is doing: greeting, troubleshooting, wrapping_up. Those constrain nothing, because they do not correspond to any fact being true.

The useful derivation runs the other way. For each consequential tool, ask: what must be true before this call is legal? Each of those facts is a state boundary.

For the returns agent:

• identifying — no verified customer yet
• order_selected — customer verified, exactly one order in scope
• eligibility_known — the return-window and condition checks have run
• resolution_proposed — a specific remedy has been stated to the customer
• closing — remedy executed or escalated

Five states, derived from four facts. refund_order exists only in resolution_proposed, because the fact it depends on — the customer has been told what will happen — is only true there.

Step 2: make transitions tool calls

The model still decides what happens next. The difference is that code decides what is on the menu.

# tools exposed in state: order_selected
check_eligibility(order_id)
ask_customer(question)
lookup_order_history(customer_id)
to_identifying(reason)        # wrong order, start over
escalate(reason)

There is no refund_order in that list. Not discouraged, not guarded by an instruction — absent.

Transitions being tool calls rather than internal bookkeeping matters for a second reason: each one carries a reason argument written by the model, and lands in the trace. Reading a run becomes reading a path — identifying → order_selected → eligibility_known → escalate — with a stated reason on each edge.

Step 3: per-state prompt fragments

The system prompt becomes a small stable core plus a section for the current state:

[core: who you are, tone, hard rules]
[state: eligibility_known]
  You know whether this return is inside the window.
  If it is not, do not propose a refund; explain the policy
  and offer the store-credit exception if the order is over 90 days.

The token saving is real but secondary. The actual gain is that a rule is written in exactly one place and applies in exactly one situation, so it can be changed without re-reading four paragraphs to find out what else it affects.

Step 4: the escape hatch people skip

⚠️ Real conversations do not respect the graph. Three turns into an eligibility check, the customer says "actually, forget that one, it's the other order I want to return." A machine with no edge for that will keep asking about the wrong order, politely, forever.

Two mechanisms cover most of it:

• A scope-reset transition available from every state. to_identifying(reason) drops the established facts and starts the fact-gathering again.
• escalate(reason) available from every state. The graph does not need an edge for every situation if it always has an exit.

Without these, the pattern produces an agent that fights the customer, which is worse than the free-form version it replaced.

What this buys

• Illegal actions are impossible, not discouraged. The compliance question stops depending on prompt wording holding up under an unusual conversation.
• Traces become reviewable objects. A named path with reasons is something a support lead can read; a forty-message transcript is not.
• Resumption is cheap. State plus established facts is a small serializable thing. Reconstructing a run does not mean replaying a transcript.
• Evals get sharp. Test one transition in isolation: given eligibility_known with the window expired, does it reach resolution_proposed with the right remedy? That is a small test with an unambiguous answer.

❌ When not to do this

Open-ended work. For a research or coding agent, the action space genuinely is "anything, in any order." A graph over that is a lie you will maintain forever, and every real task will need an edge you did not draw.

Fewer than about three consequential actions. If one tool is dangerous, a precondition check inside that tool is cheaper and clearer than a state machine wrapped around the whole agent. The pattern earns its complexity when several actions each depend on different established facts.

🔍 Migrating from a free-form agent that already works

Do not design the graph up front. Pull fifty real traces and mark every call that happened before its precondition held — the refund before the eligibility check, the address change before identity verification. Those marks cluster, and the clusters are the state boundaries worth having.

Then start with two states: before and after the single most dangerous fact. Add a third only when a trace demands it. A graph grown from observed failures stays small and stays true; a graph designed from an imagined workflow accumulates states nobody can justify.

The core move is small: stop asking the model to remember what it is allowed to do, and stop offering it the option in the first place.