What Belongs in Long-Term Memory (and What's Just Transcript)

The usual first implementation of agent memory is to embed every conversation turn and retrieve the similar ones later. It demos well and degrades badly: within a few weeks, retrieval returns fragments of old conversations that are topically similar and factually stale, and the agent starts confidently acting on decisions that were reversed months ago.

Storing everything isn't memory. It's a transcript with a search box. Memory is a curated claim about what remains true.

The test for whether something belongs

Before writing anything to long-term storage, ask: will this still be true and still be useful in a month?

Three things pass:

Stable preferences. "Prefers metric units." "Wants drafts, never sent mail." "Works in Europe/Berlin." These change rarely, apply broadly, and are cheap to carry. This is the highest-value category and the smallest.

Entity facts. "Account 8821 is on the legacy billing plan." "The payments service is owned by the platform team." Facts about durable things, ideally with a source so they can be re-verified.

Decisions and their rationale. "Chose Postgres over DynamoDB because of the reporting requirement." The rationale is the valuable half — it's what lets a future run tell whether the decision still applies when circumstances change.

What fails the test: conversational turns, intermediate results, anything derivable from a system of record, and anything true only within one run. → If the current database can answer it, don't remember it; look it up. A remembered copy of a mutable fact is a bug with a delay on it.

Write deliberately, not automatically

Automatic capture is where quality dies. Two disciplines fix most of it.

Make writing an explicit, justified act. A remember(fact, category, source, confidence) tool the agent must deliberately call — not a background process that vacuums up turns. The requirement to state a category and a source filters out most of what shouldn't be stored, in the same way that writing a commit message filters out thoughtless commits.

Store atomic claims, not passages. One fact per record, phrased to stand alone. "The user prefers morning meetings" retrieves and composes cleanly. A three-paragraph excerpt containing that preference amid other content pollutes the context and is hard to supersede later.

The contradiction problem

This is what breaks long-lived memory systems, and it needs a decision made up front rather than discovered in production.

Six months in, storage contains "prefers morning meetings" and, later, "prefers afternoons — mornings are now school run." Similarity search happily returns both. The agent picks one, or worse, tries to honour both.

Two workable approaches:

• Supersede on write. Before storing, retrieve near-duplicates and explicitly mark the old record superseded. Costs a retrieval per write; keeps the store clean.
• Timestamp everything and prefer recency at read time. Cheaper to write, and it keeps history — but every consumer must apply the rule, and one that forgets reintroduces the bug.

⚠️ What doesn't work is storing both and hoping ranking sorts it out. Recency and similarity are different axes, and generic vector search optimizes the wrong one.

Expiry as a first-class field

Give each record a review horizon at write time. Preferences: long. Project facts: the project's lifetime. Anything about a system's current state: short, if it belongs in memory at all.

A record past its horizon isn't necessarily deleted — but it should surface differently, flagged as unverified rather than presented as current. 💡 The cheapest useful version: include the write date in the retrieved text, so a two-year-old claim arrives visibly two years old and can be weighed accordingly.

What good memory looks like in practice

A store that stays surprisingly small. Dozens to hundreds of atomic claims for a user, not tens of thousands of turn fragments. Each with a category, a source, a date, and a supersession chain. Retrieval returns three to five facts, not twenty passages.

The metric that reveals health: how often retrieved memories are used in the response versus retrieved and ignored. A high ignore rate means the store is returning noise, and noise costs context and attention on every single turn.

The takeaway

Memory is an editorial function, not a storage one. The valuable question isn't how to embed the transcript — it's what deserves to outlive the conversation, said in a way that stays true. Write atomic claims deliberately, supersede contradictions on purpose, date everything, and keep the store small enough that a person could read it and agree with what's in there.