Judged on what was knowable

There is a fairness we extend to every human decision-maker without quite naming it: we judge them on what they could have known at the time, not on what we know now. A doctor who treats the likeliest cause of a patient's symptoms is not negligent because a rare cause later turns out to have been the real one. A lender who approves a sound applicant is not reckless because that applicant is later wiped out by an event no one foresaw. We hold these decisions to the standard of their moment. We accept that a good decision can have a bad outcome, and that the outcome is not, by itself, a verdict on the decision. This is one of the oldest pieces of practical wisdom we have, and automated systems are quietly eroding it.

They erode it not by deciding badly but by being reviewed badly. When an automated decision goes wrong and someone comes to examine it, they arrive already knowing how it ended. They know the loan defaulted, the flag was a false alarm, the approval was a mistake. And that knowledge is impossible to set aside. It reaches backward and recolors everything: the inputs that looked ambiguous at the time now look like obvious warnings, the judgment that was reasonable now looks careless, the path not taken now looks like the one any competent system would have chosen. The reviewer is not being dishonest. They are being human. Hindsight is not a bias you can decide to switch off; it is the water the review swims in.

The danger this creates is specific and severe. If decisions are judged by their outcomes rather than by their grounds, then a system that decided well will be condemned whenever it was unlucky, and a system that decided badly will be praised whenever it was lucky. Both are failures of justice, and both push institutions toward exactly the wrong behavior: not toward deciding well, but toward deciding defensibly in the narrow sense of avoiding any outcome that would look bad in retrospect. A reviewer armed with hindsight does not reward good judgment. They reward good luck and punish bad luck, and they teach the system to optimize for neither.

The information horizon

What protects a decision from this is a precise record of its information horizon — the line between what was known and what was not known at the moment the decision was made. This is the single most important thing a decision record can fix, and the easiest one to lose, because the horizon is invisible from the future. By the time anyone reviews the decision, the things that were unknown have become known, and the line that separated them has dissolved. The record's job is to draw that line back in: to hold, in fixed form, the exact state of available information at decision time, so that the decision can be judged against it rather than against the fuller knowledge that arrived too late to have helped.

This is a different demand from preserving the inputs, though it depends on them. Preserving the inputs tells you what the decision was made from. Fixing the information horizon tells you what the decision was made without — what was genuinely unavailable, what had not yet happened, what no diligent decider could have consulted because it did not yet exist. A record that captures only what was present, and is silent about what was absent, still leaves the reviewer free to imagine that the missing piece was there to be found. The horizon has two sides, and the record has to mark both: here is what could be known, and here, on the other side of the line, is what could not.

A good decision can have a bad outcome. Without a record of what was knowable, no one can tell the two apart afterward.

With that horizon fixed, review becomes possible in the only form that is fair: not "knowing what we now know, was this right," but "given what was knowable then, was this a sound decision." That is the question that separates a genuinely flawed decision — one that ignored or misread information that was available — from a merely unlucky one that did everything right against a future no one could see. Only the first deserves to be called a failure of the decision. The second is a failure of the world. Without a fixed horizon, the two are indistinguishable, and an institution ends up apologizing for its luck and defending its judgment in the same breath, unable to tell which it is doing.

The asymmetry that record corrects

There is a deeper reason this matters than fairness to the institution, and it concerns fairness to the affected party too, because the asymmetry cuts both ways. A horizon that is honestly recorded does not only shield a decision-maker from unfair hindsight. It also exposes the decision-maker who did have the information and failed to use it — who cannot now claim the warning was unforeseeable because the record shows it sitting right there, on the knowable side of the line, at the moment of the call. A fixed horizon is not a defense. It is a neutral fact that defends the prudent and convicts the negligent, and it can only do either if it was captured when the line was still real.

The temptation, always, is to let the horizon stay soft — to review decisions in the warm light of everything we now know and to call that thoroughness. It is not thoroughness. It is a category error dressed as diligence, the substitution of an outcome for a judgment. The discipline these dispatches keep returning to is the discipline of the contemporaneous record, and this is one more thing only a contemporaneous record can do: hold still the boundary of the knowable, so that a decision can be tried on the evidence it actually had. Judge the decision by its horizon, or judge it by its luck. There is no third option, and only one of them is just.

— Dispatches · Summit Cognitive