Who audits the auditor

The oldest problem in oversight is that oversight is also a decision. We answer the worry that a system might be wrong by placing above it a body whose job is to check — an auditor, a reviewer, an appeals panel, a supervisory layer — and for a moment the worry seems answered. But the checking body decides too. It decides whether the decision below it was sound, and that decision can be careless, biased, mistaken, or captured, exactly like the one it examined. So the question that dissolved one layer down reappears intact one layer up: who checks the checker? If the answer is another checker, the same question waits for it. If the answer is no one, then the whole edifice of assurance rests, at the top, on a decision that is trusted precisely because nothing scrutinizes it — which is the condition we built the oversight to escape.

This is why oversight that cannot itself be replayed is not a solution but a relocation. When a reviewing body examines a decision and cannot itself produce a record of how it reached its conclusion, it has not added accountability to the system; it has added an unaccountable node at the top of it, and asked us to trust that node on its authority. The trust problem has not been solved. It has been moved to a place where it is harder to see and easier to ignore, because a body whose job is to hold others to account acquires, in the doing, a presumption that it need not be held to account itself. The auditor is trusted because it audits — as if the function conferred the reliability, when the function is exactly the thing that ought to be examined most closely.

Notice what has happened. We introduced oversight to close a gap in trust, and we closed it by opening the same gap one level higher, then declined to look. The reviewer's ruling is delivered as final not because it has been shown to be sound but because it sits at the top of the chain, and the top of the chain is where inspection conventionally stops. So the most consequential judgment in the whole apparatus — the one that decides whether every judgment below it stands or falls — is the single judgment no one is positioned to check. The apparatus is strongest at the bottom, where decisions are recorded and contestable, and weakest at the top, where the decision that governs all the others is made in exactly the darkness the whole structure was meant to abolish.

An auditor who cannot be audited has not ended the question of trust; they have only carried it up to a floor of the building no one is allowed to visit.

Recursive admissibility

The way out is not an infinite tower of overseers, each watching the one below — that regress never terminates and no one has the patience or the people for it. The way out is to demand of the oversight the same property we demand of the decision: that it be replayable. A review is admissible only if it leaves a record from which the review itself can be reconstructed — what the reviewer looked at, what standard they applied, what they concluded and why — a record as open to contest as the decision under review. This is recursive admissibility: the requirement that the act of checking meet the same evidentiary bar as the act it checks, so that the reviewer, like the decided-upon, can be answered for on the record rather than trusted on their station.

What recursion buys is a place to stop that is honest. The regress of "who checks the checker" is vicious only if each checker must be checked by a higher authority — a chain that must end in someone trusted absolutely. But it terminates cleanly the moment each checker leaves a replayable record, because then the checker need not be validated by a superior; the checker can be validated by anyone, later, from the trail. The Decision Receipt the reviewer produces is inspectable by the affected party, by a court, by a successor, by a peer — none of whom sit above the reviewer in a hierarchy, all of whom can nonetheless hold the review to account by re-running it against what it claimed to have done. The tower does not need an infinite top floor. It needs every floor to be made of glass.

Where the chain can honestly stop

This reframes the whole question. "Who audits the auditor" has no good answer if the answer must be a person or a body with even greater authority, because authority is precisely what we were trying to make accountable. It has an excellent answer if the answer is a record — because a replayable account of the review does not require a superior to enforce it; it requires only that someone, anyone with standing, can open it and check. The auditor is then accountable not to a higher auditor but to the same discipline of contestability that binds the layers below. The regress ends not in a trusted final authority but in a trail that any competent stranger can walk.

So the test to apply to any oversight is not how senior the reviewer is or how independent they are declared to be, but whether their review can be replayed. An audit that produces only a verdict — passed, cleared, upheld — has moved the trust problem up a level and hidden it there. An audit that produces a record from which the audit can be re-run has kept the trust problem where it can be worked: in the open, on the evidence, answerable to whoever needs to check. Oversight does not become trustworthy by rising above scrutiny. It becomes trustworthy by submitting to exactly the scrutiny it imposes — and a reviewer unwilling to leave a record as contestable as the decision they reviewed is not the top of the accountability chain. They are the break in it.

— Dispatches · Summit Cognitive