Prompt, Tool, or Code? Where Each Piece of Logic Belongs

A rule that keeps getting violated — "always use ISO dates," "never email more than five people at once," "check the account tier first" — is almost never a prompting problem. It's logic that was put in the wrong place.

Every behavior in an agent lives in one of three homes, and each has a property the other two don't.

• Code runs every time, identically, and cannot be talked out of it.
• A tool is available when offered, and the model decides whether to use it.
• The prompt is advice the model weighs against everything else in context.

Most reliability complaints trace back to something sitting one level too far toward advice.

The placement rule

Ask two questions about the behavior.

Is it deterministic — is there exactly one right answer given the inputs? If yes, it belongs in code. Date formatting, unit conversion, ID validation, ranking by a numeric score, deduplication, "no more than five recipients." A model can do all of these and will do them correctly most of the time, which is strictly worse than a function that does them correctly every time for free.

Does it require the world? If it needs data the model doesn't have, or must cause an effect outside the conversation, it's a tool. That's what tools are for and there's rarely ambiguity here.

What's left — judgment that varies by situation, with no single right answer — is what the prompt is for. Tone. Whether an ambiguous request needs clarification. Which of three plausible categories a messy input belongs to. This residue is smaller than most system prompts assume.

The smells

"Always remember to…" in a system prompt. If it must happen every time, the loop should do it every time. Post-process the output, or enforce it in the tool handler. → A model asked to always do fifteen things will do twelve of them.

Validation instructions. "Make sure the email address is valid before sending." The send tool should reject invalid addresses. Then the rule is unbreakable and the model gets a clear error when it tries, which is more informative than a prompt line it may or may not have weighed.

Arithmetic and counting in prose. Totals, percentages, date arithmetic, "how many items match." Give it a tool or compute it in the handler.

A prompt section that grows every incident. Each new failure adds a line, the prompt reaches two thousand words, and adherence to any individual line drops. This is the accumulation pattern that most needs interrupting: the third time a rule is added, move it into code.

Worked example

Requirement: when summarizing a support thread, include the customer's account tier, never quote the customer's phone number, and keep it under 200 words.

❌ All three as prompt lines. Tier gets omitted when the thread is long. The phone number appears occasionally. Length drifts past 200 words on complex threads.

✅ Placed properly:

• Account tier → code. The handler looks it up and prepends it. The model isn't asked, so it can't forget.
• Phone number → code. Redact from the thread before it enters the context. The model cannot quote what it never saw, which is a stronger guarantee than instructing it not to.
• Under 200 words → prompt, plus a check. Length is a soft target where the model's judgment about what to cut is genuinely useful; the handler verifies and asks for a trim if it's over.

Three requirements, three different homes, chosen by what each one actually is.

💡 The redaction pattern generalizes

Anything the model must never output is best handled by ensuring it never enters the context. Instructing a model not to reveal something is a request; removing it from the input is a guarantee. This applies to secrets, personal data, internal notes on a customer record, and anything else where the failure is unacceptable rather than merely annoying.

When code is the wrong choice

The rule cuts both ways. Logic that needs judgment on every invocation belongs in the model even when it's tempting to encode it. A hundred-branch decision tree for classifying support tickets is worse than a prompt, because it fails opaquely on anything unanticipated and needs an engineer for every change.

The signal that you've over-coded: your rules need rules for their exceptions, or the branching turns on tone, intent, or context rather than on values you can test.

The takeaway

Before adding another line to your system prompt, ask whether the behavior is deterministic. If it is, it belongs in code, where it happens whether or not the model is having a good day. Prompts are for judgment. Tools are for reach. Code is for everything that must simply be true — and in most agents, that's the largest category.