The chain of custody we never built for software

Forensics built an entire discipline around a single, almost pedantic insight: a piece of evidence is only as trustworthy as the unbroken, documented record of who held it, when, and what they could have done to it while it was in their hands. A bloodied shirt, a hard drive, a vial of blood — none of it means anything in a courtroom on its own. What gives it weight is the log: collected by this person, at this time, sealed, transferred, signed for, stored here, opened there. We call that record the chain of custody, and we take it so seriously that a single unexplained gap in it can throw the evidence out entirely. Not because the gap proves tampering, but because it makes tampering impossible to rule out — and an unaccountable interval is, for evidentiary purposes, indistinguishable from one.

That is a strong rule, and it is worth dwelling on how strong it is. The evidence does not have to be shown to be corrupted. It only has to have a moment no one can account for. The burden runs the other way: the party offering the evidence must be able to say where it was the whole time, or it does not come in. We decided, as a matter of practice, that the integrity of a conclusion depends on the traceability of the things it rests on, and we were willing to lose otherwise-true evidence to keep that principle intact.

We never built the equivalent for the inputs that drive automated decisions. And those decisions now reach further into ordinary life than most physical evidence ever did.

Decisions inherit the custody of their inputs

Consider what actually feeds a consequential automated decision. Data arrives from somewhere — a vendor feed, a scraped page, a partner's export, a sensor, a form someone filled out at three in the morning. It is transformed by something — a pipeline, a join, a cleaning script written by a contractor who has since left. It is fed to a model trained by someone, on material gathered who-knows-how, under terms no one in the room can now recite. By the time a score or a flag or a denial comes out the other end, the path that produced it has been flattened into a number. Almost none of that lineage travels with the decision. The decision arrives clean and confident, like a witness who cannot tell you where he was last Tuesday but is very sure about what he saw.

This is the gap, and it is exactly the gap forensics refuses to tolerate. We would never accept a vial of blood with no record of who drew it or where it sat overnight. We accept, routinely, decisions built on data with no record of when it was captured, by whom, or through how many uninspected hands it passed on the way in. The reason we accept it is not that the lineage doesn't matter. It is that we have never been forced to look at it, because the decision does not carry it. What you cannot see, you are not made to answer for.

A decision is only as accountable as the least-accounted-for input it rests on — and most decisions never have to say what that is.

Provenance is what closes the gap. To carry the provenance of an input is to carry its custody: the captured-at, so you know how stale the world was when the decision saw it; the collector, so you know whose judgment and whose incentives shaped what was gathered; and the chain of hands it passed through before it reached the model, so that no interval in its history is simply blank. This is not metadata for its own sake. It is the difference between a decision you can stand behind and one you can only stand near — close enough to point at, not close enough to defend when someone asks the obvious question and means it.

What the receipt actually buys you

A Decision Receipt that carries the captured-at, the collector, and the chain of custody of each source is not a compliance artifact. It is the thing that lets a decision survive being questioned. When a result is contested — and consequential results are eventually always contested — the first honest question is not "what did the model conclude?" but "what was it looking at, and where did that come from?" A receipt that can answer source by source has converted a defensive crouch into a position. It can say: this input was captured at this time, by this collector, and here is the unbroken record of how it reached the decision. The contest now happens on the facts, where it belongs, rather than on the prior question of whether anyone even knows what the facts were.

And the converse is the real payoff. Where the chain is broken, the receipt shows you the break. A source with no captured-at, a transformation with no recorded author, a feed that entered the pipeline through an interval no one logged — these stop being invisible and become findings. You are no longer trusting that the inputs were sound; you are looking at exactly which ones you cannot vouch for. That is the forensic posture: not the presumption of integrity, but the ability to locate, precisely, where integrity cannot be presumed. A decision that can point to its own weakest link is more trustworthy than one that presents a uniform, untraceable confidence, because it has told you where to push.

"We logged it" is not "we can account for it"

Most institutions will say they already do this. They have logs. They have lineage tooling. They can, with effort, reconstruct where a number came from. But logging is not custody, and the difference is the whole argument. A log is something you keep for yourself, on your own terms, retained as long as you choose, written in a format only you can read, and producible — if at all — only after a long internal scramble. Custody is something you can hand to an adversary. It travels with the evidence, it is legible to the person who wants to fight you, and it does not depend on your goodwill or your retention policy to exist when it is needed.

The forensic standard was never "we have records somewhere." It was "we can account for this item's history, continuously, to a skeptic, without gaps." Those are different bars, and almost everyone who claims the first has not cleared the second. "We logged it" means the information might be recoverable by us. "We can account for it" means the information is bound to the decision and answerable to you. An automated decision that cannot meet the second bar is not necessarily wrong. It is simply not yet evidence — it is a claim wearing the costume of one, and the costume is the part that should make you suspicious.

None of this requires believing that machine-made decisions are uniquely dangerous. It requires only the same modest demand we have made of physical evidence for a century: that the thing be able to account for where it has been. We extended that demand to bloodstains and to hard drives because we understood that a conclusion is no stronger than the custody of what it rests on. The decisions that now sort applications, price risk, flag people, and route consequence deserve no smaller a standard — and they will keep arriving clean, confident, and unaccountable until we decide, as we once did for evidence, that a gap in custody is not a detail. It is the finding.

— Dispatches · Summit Cognitive