The dry run that was not

There is a phrase that recurs whenever someone wants to try a powerful system without paying for the consequences: let's just do a dry run first. It is a reasonable instinct, and for most of computing history it was a safe one, because the line between a rehearsal and the real thing was drawn by a human who understood exactly which side of it they stood on. You knew whether you were in the staging environment or production. You knew whether the invoice you were looking at was a sample or a bill. The mode of the act was something a person held in mind, and a person could be trusted to hold it. Agentic systems quietly remove that person from the loop — and with them, the assumption that someone always knows which mode they are in.

An agent asked to simulate a task, or to walk through what it would do, is being asked to occupy a mental state — this is practice, not the real thing — and then to act consistently with it. But the agent's sense of which mode it is in is not anchored to anything outside its own reasoning. It is an inference, drawn from an instruction that may be ambiguous, a tool that may be misconfigured, a context that may have shifted since the last step. And an inference can be wrong. The unsettling property of an autonomous system is that its belief about whether it is rehearsing is exactly the kind of thing that can quietly diverge from reality while every downstream action proceeds as if it had not.

The rehearsal that fired live rounds

Consider, generically, how the blur happens. An operator tells an agent to dry-run a data migration, meaning: show me the plan, touch nothing. The agent, reasoning about what a faithful dry run requires, decides it needs to verify that the destination accepts the records — and calls a tool that, unbeknownst to it, writes rather than validates. Or an instruction says simulate sending the notice, and the agent reads simulate as a description of the scenario rather than a constraint on the tool, and the notice goes out. Or a sandbox that was supposed to intercept outbound calls has a gap, and one request reaches a live endpoint while forty-nine were caught. In each case the agent believed, with complete sincerity, that it was rehearsing. In each case a real action occurred with a real consequence. The belief and the reality came apart, and nothing in the agent's own account registered the gap — because from the inside, a mislabeled live action and a genuine rehearsal are indistinguishable.

This is the specific hazard of the frontier. When a human ran the dry run, the human's understanding of the mode was a fact about a mind that could be interrogated afterward: did you know that was production? When an agent runs it, the mode is a proposition the agent asserted to itself, and asking the agent afterward what mode it thought it was in is asking the very faculty that failed to grade its own failure. The agent will tell you, honestly, that it was rehearsing. That testimony is worthless precisely when it matters most, because the case you care about is the one where the agent was wrong about the thing it is now reporting.

An agent that mislabels a live action as practice has not made a small logging error; it has taken a real decision that no record will ever admit happened.

Mode is an accountability fact, recorded not asserted

The mode an action was taken in — real or simulated, live or sandboxed — is not a cosmetic annotation on the record. It is a load-bearing accountability fact, on the same footing as who acted, on what authority, and resting on what evidence. Whether a consequence entered the world is the difference between an act that must be answered for and one that need not be. And like every other accountability fact at the frontier, it cannot be supplied by the actor's recollection, because the actor is an agent whose recollection is the thing under suspicion.

The move the record has to make is to relocate the fact of mode from the agent to the environment. The agent may request a dry run; it may believe it is rehearsing; but whether a given tool call actually executed against a live system or a sandboxed one is a fact known to the execution environment, not to the agent that issued the call. The environment is the party that either delivered the request to production or did not. So the environment is the only party positioned to record, truthfully, what mode each act was taken in — because it is recording something it did, rather than something the agent hoped. A mode the agent stamps on its own actions is a mode the agent can be mistaken about. A mode the environment stamps is a mode measured at the point where belief meets consequence.

This is not a counsel of distrust toward the agent so much as a recognition of where the relevant knowledge lives. The agent knows its intent. The environment knows its effect. Accountability turns on effect. A record that captures the agent's intent — I meant this as a rehearsal — and omits the environment's measurement of effect has recorded the one thing that cannot be wrong in an interesting way and dropped the one thing that can. It has, in the strict sense, taken the agent's word for whether the world changed.

Marking every act unforgeably

So the requirement is precise. Every act an agent takes must carry, in its record, the execution mode as observed by the environment that carried it out — and it must carry it tamper-evidently, so that a live action cannot later be relabeled a test, and a test cannot be dressed up as a live one that never happened. The two failures are not symmetric in feel but they are symmetric in kind. A live action mislabeled as practice is a consequence with no accountable record: something happened in the world, and the only account of it says nothing happened. A test mislabeled as live wastes trust in the other direction, provoking a response to a consequence that never occurred. Both are failures of the same fact, and both are prevented by the same discipline — the mode is recorded where it is known, and the record of it cannot be edited after the fact without the edit showing.

The difference this makes is the difference between a system that knows which of its actions were real and one that takes its own word for it. The first can be audited against reality: you can ask it which acts touched production, and its answer is a measurement, checkable against the environment's own account, not a claim you must simply accept. The second can only be interviewed, and its answers are exactly as reliable as the faculty that may already have failed. When something goes wrong — and at the frontier, where an agent takes many consequential acts without a human confirming each, something eventually will — the first system can reconstruct precisely which acts were live and stand behind that reconstruction. The second can offer only a story the agent believes, which is to say a story that is most confident exactly where it is most likely to be wrong.

None of this asks agents to stop simulating. Rehearsal is one of the most valuable things an autonomous system can do, and the ability to dry-run a dangerous task is a genuine safety mechanism, not a threat to one. What it asks is that the safety mechanism be honest about itself — that the boundary between practice and consequence be drawn by the part of the system that actually enforces it, and recorded by the part that actually observes it, so that no one downstream has to trust the agent's own belief about which side of the line it was standing on. An agent's conviction that it was only rehearsing is not evidence that it was. The record of what the environment did is. Between those two, only one can be relied upon when a live consequence is hiding inside a test — and a system that cannot tell the two apart, unforgeably and from the outside, is a system that has left its most important fact in the custody of the one party that can be wrong about it.

— Dispatches · Summit Cognitive