The deep-pocket problem

Something goes wrong with an automated decision, and a harm lands on a real person. What follows is almost choreographed. The vendor who sold the system explains that it was only configured and operated by the deployer. The deployer explains that it merely ran a model it did not build and could not inspect. The model-maker explains that the model behaves as its training data permitted, and that the data was assembled by someone else again. Each account is, taken alone, plausible. Together they describe a decision that harmed someone and for which, apparently, no one is responsible. The reader is invited to conclude that responsibility has dissolved — that the chain is so long and so distributed that the loss simply has nowhere to come to rest.

That conclusion is wrong, and it is worth being precise about why. The legal and economic system does not tolerate a vacuum of responsibility for a realized loss. Someone is out the money, the injury, the denied claim, the missed opportunity. That cost is not an accounting fiction; it exists, and it will be allocated. The interesting question is never whether it gets allocated — it always does — but by what rule. And the rule is not the flattering one the participants assume. It is not that the loss settles gently on whoever was most at fault, weighed with philosophical care. It is coarser and more gravitational than that.

Responsibility does not vanish, it gets allocated

Diffusion of blame across a supply chain feels, to the parties inside it, like a reduction in total liability. It is nothing of the kind. The harm is a fixed quantity; spreading the number of hands it passed through does not shrink it. What diffusion actually changes is not the size of the liability but the difficulty of locating its owner — and difficulty of location is precisely the condition under which the system stops asking who was most at fault and starts asking who can be reached. When the fault is genuinely tangled, litigation and insurance do not suspend themselves pending a clean answer. They resolve the tangle the way they always resolve intractable causation: by finding a party who is solvent, reachable, and connected to the harm closely enough to hold. This is the old, unglamorous truth that plaintiffs' lawyers have always known and defendants relearn one at a time. Loss seeks the deep pocket.

The deep pocket is not chosen because it is guilty. It is chosen because it can pay and because it is standing in the causal chain when the music stops. A diffuse harm and a solvent participant are all the system needs; the connection can be attenuated and the allocation will still find it, because the alternative — leaving the loss on the injured party — is the one outcome the system is built to avoid. So the supply chain's confident diffusion of blame does not make liability disappear. It converts a question of fault into a question of solvency, and hands the answer to whoever has the most to lose and the least ability to move it somewhere else.

There is a second variable, and it is the one the participants tend to overlook while they are busy pointing. Solvency determines who can pay. But among the solvent, something else decides who does — and that something is evidence. The party that can produce a clean, contestable account of what it did, and show that it did it soundly, can push the loss along to a party that cannot. The chain does not resolve to the richest defendant so much as to the richest defendant who also cannot explain itself.

When the music stops, the loss lands on whoever can pay and cannot explain themselves — so the cheapest liability insurance you can buy is a record of having been right.

Records decide who holds the bag

Consider two participants in the same chain, equally solvent, equally reachable. One can reconstruct exactly what its part of the decision consisted of: the inputs it received, the rules it was operating under at the time, the state that produced its output, all of it replayable by someone who does not trust it. The other can offer only assurances — a description after the fact, a policy it says it followed, a shrug where the evidence should be. When the loss is allocated between them, it will not split evenly. It will slide toward the one who cannot account for itself, because that party has given the process nothing to work with except its presence in the chain. Being unaccountable does not read as neutral. In a contest over who bears a cost, it reads as an admission — the default position for a party that cannot show it acted rightly is that it did not.

This reframes what a record is for. We tend to file the ability to reconstruct a decision under ethics, or transparency, or good governance — worthy motives that compete with the balance sheet and usually lose. But reconstruction is also, and more urgently, self-defense. A party that can prove it acted soundly holds an instrument that shifts loss away from itself and toward whoever was actually careless. A party that cannot has, in effect, volunteered to be the residual claimant on every harm it touches, the one the loss falls to when the fault cannot be cleanly assigned. The choice to keep a real account is not only a choice to be honest. It is a choice about whether, when the allocation happens, you are holding a shield or holding the bag.

Infrastructure that aligns loss with fault

None of this is a claim about any specific liability regime. Regimes vary by jurisdiction, by sector, by contract, and they will keep changing faster than any essay can track. The point is more general and more durable than any statute: whatever the regime, there is a gravity to how realized losses move, and the gravity pulls them toward the solvent and the unaccountable. You cannot repeal that gravity. But you can change where it deposits the loss, and the instrument that changes it is a record.

An accountability record — a Decision Receipt that carries the evidence actually consulted, the rules that were live at the moment, and enough state to replay the decision and let it be contested — is not, in this light, a compliance nicety. It is the mechanism that aligns where the loss lands with who was actually at fault. Without it, the allocation defaults to solvency and reachability, and the careful party subsidizes the careless one because neither can prove which was which. With it, the party that acted soundly can demonstrate as much, and the loss is free to travel to the party that did not. The record does not make anyone less liable in the aggregate; the harm is still fixed and still owed. What it does is correct the aim, so that inevitability of allocation stops being a lottery weighted by who has the most money and the worst memory.

The rational response to a loss you cannot make vanish is to make sure it lands on the party who caused it — and, if that party is not you, to be able to prove it. This is not moralizing dressed up as economics. It is economics without the moralizing: given that the cost is real, that it will be allocated, and that solvency and evidence are the two forces deciding where it comes to rest, the disciplined move is to control the one of those you can. You cannot always be the party that can pay. You can nearly always be the party that kept the record. In a world of automated decisions and long, diffuse chains, that is not the ethical high ground. It is the only ground on which you are not the default defendant.

— Dispatches · Summit Cognitive