The memory between runs

We built persistent memory into agents because we wanted them to stop starting over. A system that forgets everything between runs is exhausting to work with: it re-asks what it just learned, re-derives what it just concluded, treats every session as the first. So we gave it a place to keep things — notes about the user, summaries of past tasks, learned preferences, a running sense of what worked and what didn’t. And the moment we did, we changed what kind of thing the agent is. It stopped being a function that maps this input to this output and became something with a past. The trouble is that we gave it a past without giving that past any of the safeguards we would demand of any other record that shapes a consequential decision.

Because that is what agent memory is: a record that shapes decisions. Everything the agent carries forward — the profile it built of you, the conclusion it reached about a task, the shortcut it learned — enters the next decision as an input. And not a neutral input. Memory does not arrive at the next run flagged as a guess that might be wrong. It arrives as knowledge. The agent does not remember “on one occasion I inferred, with middling confidence, that this account prefers terse replies.” It remembers “this account prefers terse replies,” full stop, and acts on it. The inference has been promoted to a fact by the mere act of being stored. That promotion is the whole problem, and it happens silently, every time.

Memory is an input that hides as fact

Consider what is lost in the act of remembering. A conclusion is always formed in a context: from particular evidence, under particular instructions, at a particular moment, with some degree of doubt. When the agent writes that conclusion to memory, it almost never writes the context with it. It keeps the finding and discards the grounds. What survives into the next run is the verdict stripped of the case that produced it — a “this user is a security researcher” with no trace of the single ambiguous message that suggested it, a “the deploy step is safe to skip” with no record of the one run where skipping it happened to cause no harm.

This is not a storage bug to be fixed with a bigger context window. It is a category error about what a remembered thing is. A determination and the evidence for it are different objects, and when you keep only the determination you have kept the least trustworthy part — the part most prone to being wrong, and now the part with no way to be checked. The agent’s next action rests on the verdict, the verdict rests on nothing the agent can produce, and no one asked to see the grounds because the grounds were gone before anyone thought to.

An error remembered is an error compounded

A single mistaken inference is a small thing. A single mistaken inference written to memory is not, because memory is read forward. The wrong conclusion does not sit quietly in a log; it is retrieved on the next run and the run after that, and each time it is used it becomes more entrenched, because now there is a history of the agent having “known” it. What began as a fallible guess accretes the authority of a standing fact, laundered clean of its origins by repetition. It is the machine’s version of a rumor that hardens into a record — the thing everyone cites and no one can source, except that here the everyone and the no one are the same system, believing itself.

A memory without provenance is not knowledge the system earned. It is a rumor the system has decided to trust, and cannot tell you why.

The person the memory is about is in the worst position of all. A determination the agent carries about you — that you are a certain kind of customer, that your request pattern looks like abuse, that your last task ended badly — shapes how every future action treats you, and you cannot contest it, because you cannot see it. There is no notice that the record exists, no account of how it was formed, no channel to correct it. This is precisely the situation that chain of custody exists to prevent in every other domain that takes evidence seriously. We insist that a piece of evidence carry where it came from and who handled it not out of ceremony but because a claim severed from its origin cannot be tested, and a claim that cannot be tested should not be trusted to decide anything. A determination that will act on you must be able to say where it came from. Agent memory, as usually built, cannot.

Custody for what an agent knows

The remedy is not to make agents forget. It is to hold what they remember to the same standard we would hold any evidence that steers a decision. A memory should carry its provenance: when it was formed, from what sources, by which run, under what instruction, and with what confidence the agent held it at the time. Not as decoration, but as the thing that makes the memory answerable. A determination stored with its grounds can be re-examined; one stored without them can only be believed or ignored, and systems tend to believe.

What this buys is the ability to ask a question that is presently unaskable: why did the agent believe that? Today the honest answer is a shrug — it is in the memory, the memory came from somewhere, the somewhere is gone. With provenance, the answer points to a source: this conclusion was formed on this run, from these inputs, at this confidence, and here is the record you can go and check. A later action resting on the memory can be traced back through it to the moment the memory was made, and the memory itself becomes a thing that can be versioned, superseded, and corrected rather than a permanent sediment the agent keeps re-reading as truth. When the grounds turn out to be thin, the memory can be marked as such and the actions that leaned on it can be found. When the person it concerns disputes it, there is a determination to point at and a history to correct, instead of a private belief with no address.

This is the same discipline the rest of this series has argued for at the level of a single decision, now carried across the seam between runs. A Decision Receipt insists that a determination arrive with its evidence, its rules, and enough state to be replayed and contested. Persistent memory is nothing but a determination that outlives the run that made it — and it deserves exactly that treatment, because it will go on shaping actions long after everyone has forgotten it was ever a guess. An agent may remember as much as we like. What it must not do is remember without keeping custody of how it came to know — because a past no one can audit is not experience. It is just a story the system tells itself, and then acts on.

— Dispatches · Summit Cognitive