Trust and trustworthiness

There is a word we use for two different things, and the confusion between them is doing quiet damage. To be trusted is a fact about other people: it describes a state of mind in those who rely on you, and it can exist whether or not you deserve it. To be trustworthy is a fact about you: it describes a property you actually possess, the real basis on which trust would be warranted, and it can exist whether or not anyone has noticed. The two usually travel together, which is why we let one word do for both. But they are separable, and the gap between them is exactly where harm collects.

Automated systems have made the gap wide and easy to fall into, because they are unusually good at producing the appearance that earns trust while being unusually poor at possessing the property that would deserve it. A system that answers fluently, instantly, and in confident, well-formed prose triggers in us the same response we have to a competent, composed human expert. We extend trust. But the fluency that earned the trust is not evidence of the trustworthiness that should ground it. The system is believed because it sounds like something that knows, not because it has shown that it does. It has the manner of authority without the substance, and the manner is what we react to.

What trustworthiness actually is

It helps to ask what trustworthiness consists of, concretely, rather than treating it as a vague halo. A trustworthy party is one who is both willing and able to account for itself — to show, when asked, the basis on which it acted, and to let that basis be examined by someone who does not already believe it. Trustworthiness is not a feeling the trusted party emits, and it is not a track record of pleasing outcomes, which can be luck. It is a standing readiness to be checked. The trustworthy person is the one who, asked how do you know, can produce something — and does not flinch at the question, because the answer is already prepared and survives inspection.

By this measure, fluency is not merely insufficient evidence of trustworthiness; it is irrelevant to it, and sometimes a substitute for it. A system can be maximally fluent and entirely unable to account for itself — can produce a flawless-sounding answer while having no preserved basis to show, nothing that could be examined, no way to demonstrate that the confident output rests on anything at all. Such a system can be trusted, easily and widely. It cannot be trustworthy, because trustworthiness is the capacity to be checked, and a system that cannot be examined has, by definition, no such capacity. It is asking to be believed on the strength of how it sounds.

Trust is what others give you. Trustworthiness is what you can show them. A system that cannot show its work is asking for the first while withholding the means of the second.

The gap is where the failure waits

This distinction would be a matter of philosophy if the gap between the two were harmless, but it is not. The gap is precisely where the next failure lives. As long as a fluent-but-unexaminable system happens to produce good outcomes, the trust it enjoys and the trustworthiness it lacks look identical from outside; nothing forces the difference into view. The difference only appears at the moment of failure — when the confident output turns out to be wrong, and someone asks the system to account for how it reached it, and discovers that it cannot. The trust was real. The trustworthiness was never there. And the moment that reveals the absence is the moment it is too late to have wanted the thing that was missing.

So the right question to ask of any system we are tempted to rely on is not whether it inspires confidence — they all do, that is what they are built to do — but whether it could demonstrate that the confidence is warranted, on demand, to a skeptic. Can it show the basis it acted on? Can that basis be examined by someone who starts out unconvinced? If the answer is no, the system may be perfectly trusted today, and the trust is simply a loan against a future failure, drawn on a trustworthiness that does not exist. The fluency is the interest payment that keeps the loan from being called.

This is where the record does its essential work, and why it is not a compliance afterthought but the substance of the thing. A decision record — the evidence actually consulted, the rule actually in force, the state from which the conclusion actually followed — is trustworthiness made visible. It is the system's capacity to be checked, rendered into something that can be produced and examined. A system that ships such a record with its decisions is not asking to be believed on its manner. It is offering the means of its own verification, which is the only form trustworthiness has ever taken. A Decision Receipt is, in this sense, the difference between a system that sounds trustworthy and one that can prove it.

And the record keeps trust honest in the other direction, too. It closes the gap that fluency opens, by making it possible to withhold trust from a system that has not earned it and extend it to one that has — to align what we believe about a system with what the system can actually show. Trust, left to fluency, drifts free of trustworthiness and floats on impression. The record tethers it back. It is how trustworthiness becomes something a system has rather than something it performs, and how trust becomes something earned rather than something granted to whatever speaks most smoothly.

— Dispatches · Summit Cognitive