The weakest link in the chain

No consequential automated decision is made by a single thing anymore. By the time a number arrives at the point where someone acts on it, it has passed through a chain of hands. A vendor sold a score. A model that nobody in the building wrote produced a probability. A data feed delivered a field whose meaning was settled in a contract three companies away. Each link did its small part and passed the result along, and at the end of the line a person or a system treated the output as a fact about the world. The decision looks like one act. It is the visible tip of a supply chain.

We are comfortable thinking about supply chains for physical things. We know that a finished product can be undone by one contaminated ingredient, one supplier who cut a corner, one component sourced from a place nobody inspected. We accept that the quality of the whole is hostage to the quality of its least trustworthy part. The same logic governs decisions, and we are much less comfortable applying it there, because the inputs to a decision are invisible and the failures are silent. A bad bolt announces itself. A bad input keeps its appearance and quietly poisons the conclusion.

The thesis is simple and unforgiving. A decision is only as accountable as its least-accountable input. You can build a flawless record of everything that happened inside your own walls — every rule, every step, every timestamp — and one opaque link upstream will still hollow it out. If a score arrived from a supplier who will not say how it was made, then the account of the decision that used it contains a hole the exact size of that score. You can describe what you did with the number. You cannot account for the number.

You cannot reconstruct what one supplier will not let you see. The chain is accountable end to end, or it is not accountable at all.

The hole is shaped like the input you cannot see

Consider what it means to stand behind a decision. To stand behind it is to be able to answer, when challenged, how it came to be — what was considered, what rules were applied, why this outcome and not another. That answer is a chain of justification, and a chain of justification has the same property as a physical chain: it carries no more load than its weakest link will bear. If one link in the reasoning rests on a bought score that cannot be examined, then the whole justification rests on it, and the whole justification can be pulled apart there. The polish of the surrounding links does not help. It may even hurt, by lending borrowed credibility to the one place that has none.

This is why provenance cannot be a property you assign to a decision after the fact. Provenance is the documented answer to where did this come from, and through whose hands. For the parts you built, you can answer that yourself. For the parts you bought or fetched or borrowed, you can only answer as well as your supplier lets you. A vendor model that returns a verdict but not its sources hands you a result you can use and cannot defend. An unlogged feed delivers a value that was true at some moment you cannot name, derived by a process you cannot inspect. Each of these is a link you do not control, and a decision built on them is accountable only up to the first such link and no further.

The uncomfortable consequence is that you cannot buy your way out of this with care on your own end. Diligence inside the building does not repair an opaque input. If the supplier will not show the work, no amount of internal rigor reconstructs what was withheld — the missing piece is missing, and labeling the gap as proprietary does not fill it. Reconstruction has a hard floor: you can replay only what you were given enough to replay. Where the inputs stop, the account stops, and everything past that point is taken on faith dressed up as fact.

Accountability is a property of the chain

If the weakness lives in the links, then the remedy is not to inspect the final decision harder. It is to make accountability a requirement of the chain — a condition that every link must satisfy before its output is allowed to flow downstream. The question to ask of each supplier is not whether they are reputable, but whether they have handed you what you would need to answer for their part when someone contests it. The sources behind the score. The state behind the feed. Enough of the model's own provenance that its output is something you can stand on rather than something you have to hope about.

This reframes a procurement decision as an evidentiary one. When you accept an input you cannot account for, you are not just taking on a quality risk; you are accepting, in advance, that a portion of every decision built on that input will be undefendable. That may be a fine trade for low-stakes work. It is a poor one wherever a decision can be challenged, where someone affected has standing to ask how it was reached, where a Decision Receipt is supposed to carry the weight of an explanation. In those settings the opaque link is not a convenience with a footnote attached; it is a structural defect that the rest of the system inherits.

The discipline this asks for is to treat the boundary of your own visibility as the real boundary of your accountability, and to refuse to pretend otherwise. Map the chain. Name the links you do not control. Demand that they become legible, or carry the decision knowing exactly where its account runs out. The honest position is not that the chain is sound because the parts you can see are sound. It is that the chain is exactly as sound as its least transparent member, and that the work of accountability is the work of finding that member and either fixing it or admitting it. A record that hides the weak link does not strengthen the chain. It just chooses where it will quietly break.

— Dispatches · Summit Cognitive