A model owes you its sources

Ask a model a question and it will answer in the register of someone who knows. The sentences are fluent and composed; the tone is settled; there is no visible seam between what it was told and what it inferred, between the part it read somewhere and the part it made up on the way to sounding sure. This fluency is the product working as designed, and it is also the precise reason you should be careful. A confident sentence and a justified one are made of different stuff, and from the outside they are indistinguishable. The only way to tell them apart is to see what the answer was built from. A model that will not show you that is asking you to take its composure for proof.

The distinction I want to hold onto is old and not at all about machines. An assertion is a statement offered on the authority of the one making it: trust me, this is so. Evidence is a statement offered together with the grounds that would let you check it for yourself: here is what I drew on, go look. The difference is not rhetorical polish — an assertion can be more polished than evidence, often is. The difference is whether the recipient is given anything to do other than believe. Evidence hands you a task. An assertion hands you a conclusion and hopes you will not ask for one.

By that standard, a model output stripped of its sources is an assertion, no matter how it carries itself. It may be a correct assertion. It may be the product of a system that is right far more often than the people it replaced. None of that changes its category. Without the sources — without the actual material the answer drew on — there is nothing in your hands to examine, nothing to confirm or to break. You have been told a thing by something that sounds authoritative. That is the whole of the transaction. Authority that cannot point past itself is just confidence with better diction.

The debt runs to the person, not the auditor

It is tempting to file provenance under compliance — a thing you produce for regulators, retain for auditors, surface when someone with a subpoena asks. That framing gets the direction of the obligation wrong. The sources are owed first to the person the decision lands on. When a system denies a claim, flags an account, ranks an application below the line, the person on the receiving end has a standing interest in knowing what the decision actually consulted — not a summary of the factors, not a category label, but the specific material that moved the outcome. They are the party with the most to lose and the least access, and the obligation to show the sources is owed to them before it is owed to anyone holding a clipboard.

And it has to be the real sources, in the order they were consulted, because order is part of the meaning. A figure read before the context that qualifies it does different work than the same figure read after. A record that lists what was consulted but flattens the sequence has discarded information that bears directly on whether the decision was reasonable. Provenance is not a bibliography appended at the end to look rigorous. It is the trace of what the decision actually leaned on, in the sequence it leaned on it, preserved well enough that someone else can walk it.

Show me what you read, in the order you read it, and I can argue with your decision. Show me only your conclusion, and all I can do is submit to it.

Notice what this gives the affected person that a summary never can: a place to stand. If the sources are visible, they can point to one and say this is stale, or this is about a different person, or this was given weight it does not deserve. That is contestation, and contestation requires a surface. Provenance is that surface. Withhold it and you have not merely declined to be transparent — you have removed the only ground on which the decision could be challenged, and left the person with nothing to do but accept it or rage at it, which from the institution's point of view amount to the same harmless thing.

Withholding is a request for unearned trust

So consider what a system is actually doing when it returns an output and keeps its sources to itself. It is asking you to trust the conclusion on the strength of the conclusion alone. But trust, the kind worth extending, is earned by exposure — by a willingness to be checked and to survive it. A system that shows its sources is making itself checkable; it is putting its grounds where you can reach them and inviting you to test whether they hold the weight. A system that withholds them is asking for the deference owed to something that has been verified, while declining to be verifiable. That is not a neutral omission. It is a request for trust the system has not done the work to deserve.

There is a tidy objection here, and it is worth meeting head on: sometimes the sources are genuinely too large, too tangled, too sensitive to expose in full. Granted, in part. But almost always this is offered as a reason to show nothing, when the honest version is a reason to show the relevant slice — the specific material that bore on this decision, for this person, rendered so they can see what touched their case. The cost of provenance is real, and it is mostly a cost the system would rather not pay. The fact that it is inconvenient to show your sources has never been a reason you do not owe them. It is, more often, the reason the owing matters.

None of this is hostility toward models. A model that carries its sources — that hands you the material it drew on, in the order it drew on it, so you can confirm the answer or take it apart — is more trustworthy than any oracle that merely sounds sure, and trustworthy in the only way that survives contact with a skeptic. The point is not to distrust the answer. It is to insist on the thing that converts an answer into evidence. Ask any system that decides something about you the question its fluency is built to forestall: what did you actually read, and will you show me? If it shows you, you are holding evidence. If it only repeats itself, more confidently, you are holding an assertion — and you are entitled to treat it as exactly that.

— Dispatches · Summit Cognitive