Your Assistant Still Thinks You're Vegetarian Because of One Dinner Party. Build the Memory Write Gate Before the Retriever.
Most agent memory bugs are write bugs: a one-off remark gets stored as a permanent fact, and no retriever can fix what should never have been saved. Here is a four-question gate that decides what goes in, with scope, provenance and an expiry on every entry.
In March you asked your personal assistant agent to find restaurants for a dinner party, adding "one of the guests is vegetarian, so keep it meat-free." In September it is still filtering steakhouses out of every lunch suggestion. The retriever is doing its job perfectly. The problem is that the sentence should never have been written to memory as a fact about you.
When a team starts adding long-term memory to an agent, most of the effort goes into retrieval: embeddings, ranking, how many memories to inject. The write side gets one line in the system prompt, something like "save anything important about the user." That line is where the damage happens, and it is cheap to fix.
🔍 Why write bugs look like read bugs
A bad memory surfaces at read time, so it gets debugged at read time. Someone tunes the similarity threshold, or adds a recency weight, and the steakhouse problem goes quiet for a week before coming back in a different form.
Look at what was actually stored:
memory: "User is vegetarian."
Three things went wrong in that one line:
- Wrong subject. The fact was about a guest, not the user.
- Wrong scope. It applied to one event, and was stored as a standing preference.
- Lost provenance. Nothing records that it came from a single remark in March, so nothing can later judge how much to trust it.
No retrieval strategy can recover information the write step threw away. If the stored record says "User is vegetarian," a perfect retriever returns "User is vegetarian."
🧩 The write gate: four questions per candidate memory
Instead of letting the model save freely, have it propose candidate memories and pass each through a gate. The gate can be a second, small model call or a structured step in the same turn. Either way, every candidate must answer four questions before it is stored.
1. Who is this about?
The user, a named third party, a project, or the current task. Facts about other people and one-off tasks belong in task context, not in the user's profile. "One guest is vegetarian" is about a guest at one event, so it stays in the dinner-party thread and dies with it.
2. Did the user say it, or did the agent infer it?
A stated fact ("I'm allergic to shellfish") and an inference ("user seems to prefer morning meetings, based on three bookings") deserve different treatment. Inferences can be stored, but flagged as such, and they should never override a stated fact.
3. How long should it live?
Give every memory an expiry class at write time:
- Standing: stated preferences and identity facts. No automatic expiry, but reviewable.
- Seasonal: true for a stretch of time ("training for a marathon", "working on the Q3 audit"). Expires unless reconfirmed.
- Event: true for one occasion. Usually should not be written to long-term memory at all.
4. What does it replace?
Before inserting, search for existing memories on the same subject and key. If the user said "I've stopped drinking coffee," the old "takes coffee black" entry must be superseded, not left sitting next to the new one for the retriever to pick between.
📦 What a gated record looks like
The output of the gate is a structured record rather than a free-text sentence:
{
"subject": "user",
"key": "diet",
"value": "no shellfish (allergy)",
"source": "stated",
"evidence": "conv 2026-03-14: 'I'm allergic to shellfish'",
"lifetime": "standing",
"review_after": "2027-03-14",
"supersedes": null
}
And the dinner-party remark, run through the same gate:
{
"decision": "reject",
"reason": "subject is a third party; scope is a single event",
"route_to": "task_context"
}
The subject + key pair is what makes question 4 workable. Superseding "user / diet" is a lookup. Superseding a pile of free-text sentences is a fuzzy search that will miss.
⚙️ A minimal implementation
Here is the flow in pseudocode, run at the end of a conversation turn:
candidates = propose_memories(turn) # model lists possible facts
for c in candidates:
verdict = gate(c) # the four questions
if verdict.decision == "reject":
task_context.add(c) if verdict.route_to else None
continue
existing = store.find(subject=c.subject, key=c.key)
if existing:
store.supersede(existing.id, c) # keep old row, mark inactive
else:
store.insert(c)
Two details matter more than they look:
- Supersede, do not delete. Keeping the old row, marked inactive, means you can answer "why did the assistant think that?" and undo a bad overwrite.
- Default to reject. The gate's prior should be "this is not worth remembering." An assistant that remembers too little asks the user again, which is a mild annoyance. One that remembers wrongly acts on it silently, often for months.
🔁 Let expiry do the cleanup
The lifetime and review_after fields let the system tidy itself up. A seasonal memory past its review date isn't injected as fact. Instead the agent can confirm it in passing: "Are you still training for the marathon? I've been blocking early mornings for runs." One confirmation resets the clock. Silence or a "no" retires it.
This is also where inferred memories earn or lose their place. An inference that the user never contradicts across several review cycles can be promoted. One that the user corrects once gets retired and, ideally, the pattern that produced it is logged so you can see which inferences your agent gets wrong.
✅ A quick audit of an existing memory store
If your agent already has memory, you can check whether you have a write problem in an afternoon. Pull a sample of stored entries and mark each one:
- ❌ Is about someone other than the user, stored in the user's profile
- ❌ Describes a single event as a standing fact
- ❌ Has no trace of where it came from
- ❌ Contradicts another active entry on the same topic
- ❌ Is an inference with nothing marking it as one
If a meaningful share of your sample hits any of these, retrieval tuning will not help. The fix is upstream.
The point
Memory quality is decided when something is written, not when it is read. A retriever can only rank what you gave it. Put a gate on the write path that asks who, stated or inferred, how long, and what it replaces, and store the answers as fields rather than prose. The steakhouse problem then goes away because the wrong fact was never stored, not because the ranking hides it.