Name what it must not decide

Write the no-go list before you write the deployment plan. Enumerate, in advance and on the record, the class of decisions the system is not permitted to make on its own — and build it to refuse those and hand them to a person, rather than discovering the boundary the way most organizations do, which is by watching the system cross it. This is not a paranoid exercise. It is the ordinary consequence of noticing that a system with authority but no stated limit does not stay where you put it. It expands until someone objects, and by then it has been deciding things no one ever agreed it should for longer than anyone realizes.

The instinct is to define what to automate and treat everything else as out of scope by omission. That is the mistake. Out of scope by omission is not a boundary; it is an absence, and absences get filled. The decisions the system must not make are the ones you have to name explicitly, because those are the ones that will otherwise be made silently, by default, by a system that is available and usually works. The limit has to be written down and built in, or it does not exist.

The scope that creeps

Consider how it actually happens, because it never happens as a decision. You deploy a system for a narrow, well-understood task — one kind of judgment, on one population, with stakes everyone has examined. It works. Because it works, and because it is right there, someone points it at an adjacent decision. The adjacent decision is a little higher-stakes, or a little more novel, or a little harder to reverse, but it is close enough to the original that pointing the system at it feels like using a tool for what it is for. No one convenes a meeting to expand the mandate. The expansion is a series of small, locally reasonable uses, each one a short step from the last, none of them the moment anyone would flag.

And the steps keep going, because nothing in the path says stop. The system does not know it has left the territory it was validated for; it produces an output for the new decision exactly as confidently as for the old one, because producing outputs is what it does. The people using it experience continuity, not transgression — the interface is the same, the latency is the same, the answer arrives. So the boundary of the system's authority is discovered only after it has been crossed, usually when a decision that should never have been automated goes wrong in a way that makes someone ask who approved this — and the answer is that no one did, and no one refused, because the question of where the authority ended was never asked.

The deeper point is that the extent of what should be automated is a governance decision, and letting it emerge from what the model happens to handle well is abdicating that decision to the model. Capability is not authority. That a system can produce a plausible answer to a question tells you nothing about whether it should be the thing that answers it. The boundary between the decisions a system may make alone and the decisions it must escalate is a choice about stakes, reversibility, and who bears the consequence — the kind of choice a governance function exists to own. When it is unwritten, it is not absent; it has simply been delegated, by default, to whoever last found the system convenient.

A system with no stated limit does not stay in its lane; it discovers the lane's edge by driving off it, and calls the wreck a new capability.

Write the no-go list

So write it. Enumerate the decisions the system must not make autonomously, and enumerate them by the properties that make a decision unfit for silent automation. By stakes: above some threshold of consequence, a human decides. By irreversibility: a choice that cannot be undone, or undone only at great cost, is not one to hand a system that cannot be argued with after the fact. By novelty: a case unlike anything the system was validated against is exactly the case it should refuse, because its confidence there is unearned. By rights-implications: a decision that materially affects a person's standing, access, or livelihood is one they are entitled to have a human accountable for. And by explicit category — some decisions belong on the list simply because your institution has decided, as a matter of policy, that a machine will not make them, and that is a sufficient reason.

Then encode the list as a hard escalation, not a guideline. This is the load-bearing distinction. A guideline that says the system should defer sensitive cases to a human will be observed exactly as often as deferring is convenient, which under deadline is rarely. The control that works is the one wired into the path: when a decision meets a no-go criterion, the system cannot proceed on its own — it stops and routes the case to a person with the authority to decide it. The refusal is not advice the operator may override at a dashboard; it is the system's behavior. The whole value of naming the boundary in advance is lost if crossing it remains a matter of someone remembering not to.

And log the refusals as first-class events. When the system hits the boundary and hands off, record that it did — the case, the criterion it tripped, the human it escalated to, the moment. This is not bookkeeping. The refusals are as important to the record as the decisions, because they are the evidence that the limit is real and being enforced, and because their pattern is diagnostic: a boundary that is never hit was drawn in the wrong place or is not wired in, and a boundary hit constantly tells you the system is deployed against decisions it was never meant to face. You cannot govern a limit you cannot see being enforced. Make the escalations legible the way you would make any consequential decision legible — as something that happened, on the record, that an affected party could later examine.

A boundary that moves deliberately

Now the honest part, because the no-go list is not a monument. It should move. As a system accumulates a track record on a class of decisions, the case for letting it act alone on some of them strengthens; as stakes rise or the world shifts, decisions that were safe to automate become ones that are not. A boundary that never changes is not principled, it is merely stale. So the list is meant to be revisited — but revised deliberately and on the record, not eroded silently in the course of everyday use. The difference between those two is the entire discipline. A boundary that moves because someone examined the evidence and decided, in writing, that it should, is governance. A boundary that moves because the system kept getting pointed at slightly harder cases until the old line was quietly behind it is the failure this directive exists to prevent, wearing the costume of progress.

Which is why the revision has to look like the original: a deliberate act, with a rationale, an owner, and a date, that an affected party or a regulator could later inspect and contest. Expanding what a system may decide alone is a decision of exactly the kind the list governs, and it deserves the same scrutiny as the decisions themselves — more, because it changes what happens to every case that follows.

And do not be deterred by the fact that drawing the line is genuinely hard. It is. Stakes are continuous, not binary; novelty is a matter of degree; reasonable people will disagree about where a given decision falls. But the difficulty is precisely the argument for doing it consciously rather than by default, because the line gets drawn either way. If you do not draw it deliberately, it is drawn for you — by what the model happens to handle, by what was convenient under pressure, by the accumulated drift of a hundred small uses no one flagged. A hard line, drawn imperfectly but on purpose and open to revision, is more defensible than a soft one that no one ever chose. Name what the system must not decide, build it to refuse, and keep the list where everyone can see it move — or the system will decide that for you, and it will not tell you when it did.

— Dispatches · Summit Cognitive