The silence of the satisfied

An institution running an automated system tends to know it is working the same way it knows the weather is fine: nobody is complaining. The queue of grievances is short, the escalations are rare, the people it serves go on with their lives without raising a hand. From inside, this reads as a verdict. If the system were unfair, surely the harmed would say so, and they mostly are not, so the system must be fair. The reasoning feels airtight, and it is one of the most reliable ways an institution can be wrong about itself, because it rests on treating silence as testimony when silence is the one thing that testifies to nothing at all.

The trouble is that complaints are a sample, and a badly biased one. The people who complain are not a cross-section of those the system touches; they are the particular subset who noticed a harm, understood it as a harm, knew where to direct an objection, and had the standing, energy, and confidence to lodge it. Every one of those filters removes people. The person who never learned they were harmed cannot complain. The person who suspects but cannot prove it does not bother. The person who assumes the outcome was their own fault blames themselves instead of the system. And the person the system served well has nothing to say at all. What reaches the institution is not the shape of its performance; it is the shape of who happened to object, which is a different and far narrower thing.

Two kinds of silence

It helps to separate the silence of the satisfied from the silence of the harmed, because they look identical from the institution's chair and mean opposite things. The satisfied are silent because nothing went wrong; their silence is genuine information, and reading it as approval is roughly correct. But the harmed are also, very often, silent — and their silence carries the same surface appearance while meaning the reverse. They are silent because they never found out, or found out too late, or could not tell whether the outcome was a decision or a fact, or had no route by which an objection could travel. When the institution pools these two silences together and counts them all as contentment, it commits a specific error: it treats the absence of a signal as the presence of a good one. The people it has failed most completely — the ones who cannot even locate the harm done to them — are precisely the ones who add most to the reassuring quiet.

You cannot hear the decisions that went wrong for people who never learned a decision was made. Their silence is not a verdict of fairness. It is the sound of a record that was never kept.

This is why an institution that governs by complaint volume drifts toward serving the articulate and losing track of everyone else. Its feedback loop is tuned to the people who can push back, and it optimizes, quietly, to reduce their pushback — which is not the same as reducing harm, and can diverge from it entirely. A system can grow steadily less fair to a population that never complains while its complaint metrics improve, because the improving metric is measuring the wrong thing. Quiet is being manufactured at the top of the distribution and mistaken for justice across the whole of it.

Reading the record, not the objectors

The way out is not to solicit more complaints, which only re-weights the same biased sample. It is to stop relying on who objected and start reading the decisions themselves. A record of what the system actually did — every decision, not only the contested ones — is the one instrument that does not depend on the affected party noticing, understanding, or protesting. It contains the outcomes of the silent alongside the outcomes of the loud, weighted by nothing but the fact that they happened. Read in the aggregate, such a record can show what no volume of complaints ever will: that a particular group is being declined at a rate no one has objected to, that a particular pattern of harm recurs among people who never came forward, that the system's real behavior and its complaint profile have nothing to do with each other.

This is the difference between listening and looking. Listening for complaints tells you what the objectors experienced. Looking at the record tells you what everyone experienced, including the great majority who will never say a word. An institution that only listens is at the mercy of the biases in who speaks; an institution that also looks can find the harm that generates no sound. And it can do so before the harm accumulates to the point where even the silent are driven to complain — which is usually the point at which the damage is already done and merely, at last, audible.

Reference the standard from Admissible Reality: a system is accountable not when its subjects are content but when its decisions can be examined on their own terms, independent of whether anyone objected to them. Contentment is a feeling, unevenly distributed and unreliably reported. A record is a fact, kept for the silent and the loud alike. The satisfied will not thank you and the worst-harmed cannot warn you, so the only honest account of what a system is doing to the people who never speak is the one written down at the moment each decision was made — read not by who complained, but by what happened.

— Dispatches · Summit Cognitive