The goal was the decision

Watch an autonomous agent work and you will be tempted to look in the wrong place for the decision. The steps are mesmerizing — the tool calls, the branches taken and abandoned, the little chain of reasoning it narrates as it goes. It looks like a mind deciding, and so it looks like the place where accountability lives. But almost none of what you are watching is a decision in the sense that matters. Each step is a means, chosen to serve an end the agent did not choose. The end was chosen before the agent moved. It was handed down in the goal, and the goal is the decision that decided everything else.

This is easy to miss because the goal is so quiet. It arrives as a single line — an objective, a target, a thing to optimize — and then it disappears into the machinery, and what we see afterward is motion. The motion is loud and the objective is silent, so we instrument the motion. We log every action, capture every intermediate thought, build elaborate traces of the trajectory. And we come away feeling we have accounted for the system, when in fact we have accounted for its obedience. We have recorded, in exhaustive detail, how faithfully the agent pursued a goal we never wrote down.

The choice is made before the agent moves

Consider an agent set to reduce cost across some operation. The instruction is unobjectionable; every organization wants it. The agent begins, and every step it takes is defensible on its own terms. It renegotiates a contract — sensible. It consolidates a redundant process — prudent. It trims a margin here, defers a purchase there, reroutes a workflow to a cheaper path. Fifty small choices, each one a thing a reasonable person would sign off on if shown it in isolation. And at the end there is a harm — a supplier starved, a safety buffer quietly spent, a population underserved — that no single step authored and that everyone, examining the steps one by one, would defend.

The harm was not authored at the step level. It was authored at the goal level, the moment someone decided that cost was the thing to reduce and declined to say what must not be sacrificed to reduce it. The agent did exactly what it was told. That is the unsettling part: there is no rogue action to find, no step where it went wrong, because the outcome was already implied in the objective before the first tool call. You can audit the trajectory forever and never locate the decision, because the decision is not in the trajectory. It is upstream of it, in a sentence.

This is the old lesson about metrics, arriving now with an actuator attached. A measure, made into a target, stops describing the world and starts deforming it — the target rewrites the behavior that was supposed to be measured. When a person held the metric, the deformation was slow and negotiated, tempered by a hundred unstated judgments about what the number was for. An agent holds the metric with no such temperance. It optimizes the letter of the goal with a literalness no human colleague would, and it does so at machine speed, across more decisions than anyone will read. The metric does not merely become the target. The target becomes the whole of the agent's world.

You do not need to watch an agent's every step to hold it to account. You need to know, exactly, what it was told to want — and who decided it should want that.

An objective has provenance

If the goal is the decision, then the goal is what the record must capture — not as background, not as a caption on the trace, but as a first-class, provenanced fact. An account of an agent's conduct that does not contain the objective it was pursuing has omitted the one input that determined the rest. It has described a river in great detail and left out the slope.

Capturing the goal means capturing four things, and capturing them the way a decision record captures anything that has to survive contestation: fixed, attributable, and legible after the fact. The exact wording the agent was given, because phrasing is not cosmetic — reduce cost and reduce cost without degrading service below the agreed floor are different instructions that produce different worlds. The constraints and guardrails set on it, in full, so that the question of what the agent was forbidden to trade away can be answered from the record rather than reconstructed from wreckage. The success measure it was optimizing, since that is the thing the whole run was bent toward. And the provenance of all of it — who set the objective, when, and on what authority — so that the decision can be attributed to a person rather than diffused into a thousand innocent steps.

That last element is the one that turns instrumentation into accountability. A trajectory has no author; it is a consequence. An objective has an author — someone chose it, chose its wording, chose what to leave unconstrained. Recording the objective with its provenance means there is a name attached to the real decision, a party with standing to have set it and standing to answer for it. Without that, responsibility does exactly what water does: it spreads out, finds the level, and vanishes into the ground between the steps.

Governing the goal

All of which points to where review actually belongs. We have built our oversight instincts around the trajectory — approve this action, inspect that reasoning, put a human near the individual acts. That is not worthless, but it is aimed downstream of the decision. The consequential moment already happened, in the setting of the objective, before any action existed to approve. Governing an agent by reviewing its steps is like governing a policy by proofreading its press releases: you are scrutinizing the fluent, visible output of a choice made somewhere you are not looking.

The objective is where authority should be exercised and where accountability should attach. Who is permitted to set an agent a goal, and how broad a goal, is a governance question of the first order — a narrow, well-bounded objective from a named author is an accountable act; a sweeping optimize this from nowhere in particular is an abdication dressed as delegation. The wording deserves the scrutiny we currently spend on individual actions. The constraints deserve to be an explicit, recorded decision rather than an absence no one noticed. And the whole thing deserves an author of record, because a goal without an author is a harm without a party to answer for it.

A system that logs every action but not the goal has made a category error about what it was accounting for. It has instrumented the means with great fidelity and lost the decision entirely. The steps were never the choice; they were the execution of a choice already made. The record has to reach upstream, past the motion, to the quiet sentence that set the motion going — because that sentence is the decision, its author is the answerable party, and everything the agent visibly did was only the sentence coming true.

— Dispatches · Summit Cognitive