The market for unaccountable decisions

The most useful thing ever written about accountability was written about used cars. In a short 1970 paper, George Akerlof asked why the market for secondhand cars is so thick with bad ones — why, if you buy a car a week old, you can expect to pay far less than its owner paid, far more than the drop in the car's actual quality would justify. His answer was not about cars at all. It was about information. The seller knows whether the car is sound or a lemon; the buyer, standing in the lot, does not, and cannot cheaply find out. And when quality cannot be observed before purchase, something perverse happens to the price — and then, through the price, to the quality itself.

The logic is worth walking slowly, because it is exactly the logic now governing the market for decisions. If a buyer cannot tell a good car from a bad one, the most he will rationally pay is a price set by the average quality of the cars on offer — he is buying a lottery ticket, and he prices it accordingly. But that average price is an insult to the owner of a genuinely good car, whose vehicle is worth more than the pooled average. So the good owner withdraws; he keeps the car, or sells it privately to someone who can inspect it. His exit lowers the average quality of what remains on the lot, which lowers the price a rational buyer will pay, which drives out the next-best cars, and so on down. Akerlof called this adverse selection. In its extreme form the market unravels completely: the good is driven out so thoroughly that only lemons are left, or no market exists at all. Quality that cannot be verified is quality that cannot command a price, and quality that cannot command a price stops being supplied.

The decision is the used car

Now change the good. Instead of a car, put a decision on the lot — a credit approval, a fraud flag, a candidate screen, a benefits determination, a triage. The buyer is whoever must rely on that decision: the person subject to it, the institution deploying it, the regulator who will one day ask whether it was sound. And ask Akerlof's question: can the buyer verify quality before he relies on it? Overwhelmingly, no. Two decisions arrive looking identical — the same confident output, the same clean interface, the same air of having been produced by something sophisticated. One rests on good evidence, followed a defensible rule, and can be reconstructed and defended. The other is a guess wearing the same clothes. From the outside, at the moment of reliance, they are indistinguishable. Soundness is exactly the hidden quality Akerlof described, and the market prices it the same way: at the average, which is to say, at nothing.

The consequence is the used-car lot all over again. If the buyer of a decision cannot tell the accountable one from the unaccountable one, he will not pay the premium the accountable one costs to produce — because producing an account is not free, and the vendor who skips it can always undercut the vendor who keeps it. The careful vendor, unable to command a price for care no one can see, faces the good owner's choice: eat the cost, or stop paying it. Enough of them stop, and the average quality of decisions on offer falls, and the market drifts toward the cheapest thing that still looks like a decision. Opacity is not chosen because anyone prefers it. It wins the way lemons win — because verification was impossible, and so care went unrewarded, and so care withdrew.

Soundness that cannot be verified cannot command a price. And a good that cannot command a price stops being supplied — not because anyone wanted it gone, but because no one could pay for it.

What broke the used-car market's fall

The reason we still have a used-car market — the reason it did not unravel to nothing, as Akerlof's cleanest model says it should — is that people built institutions to solve the information problem the market could not solve on its own. The warranty is one: a seller who offers to bear the cost of a defect is credibly signaling that he does not expect one, because a lemon-seller could not afford the promise. Certification is another: a trusted third party inspects the car and issues a report the buyer could not produce himself, and that report is a substitute for the buyer's own missing sight. Brand and repeat dealing are a third: a name that expects to sell you a car again has staked something on this one being sound. Each of these is a mechanism for making the hidden quality visible enough to price — for letting the good owner prove his car is good, so he can stay in the market.

This is the whole shape of the escape, and it is worth naming plainly because the market for machine decisions is at exactly the point where it needs to build these institutions and mostly has not. The escape from the lemons trap is never exhortation — you do not fix adverse selection by asking sellers to be more honest, because the honest ones are already being punished. You fix it by making quality verifiable: by warranty, by certification, by a record a hostile third party can inspect. Translated into the vocabulary this series uses, the escape is a decision that carries its own account — a record of what it rested on and how it was reached, which a buyer, a regulator, or the affected party can examine without taking the vendor's word. That record is the certificate. It is the thing that lets the careful vendor prove his care and command the premium it deserves, which is the only thing that keeps care in the market at all.

There is a market being born here, and it is the market Akerlof's paper predicts must be born wherever quality goes hidden: a market for assurance. Auditors who can attest that a class of decisions was sound. Insurers who will price the risk of a decider being wrong, which they can only do if the decisions leave a record they can examine. Certifiers who stand between the vendor and the buyer and lend the buyer sight he does not have. These are not compliance overhead grafted onto a functioning market. They are the functioning market — the institutional machinery that lets a good whose quality is invisible be traded at all, instead of collapsing into the cheapest indistinguishable substitute. The alternative to a market for assurance is not an unregulated market for decisions. It is Akerlof's terminal case: a lot with nothing on it but lemons, priced as lemons, because everything better has quietly left.

The decision that cannot be verified is the used car whose hood will not open. You can buy it. You will pay the lemon price, because that is the only rational price for a thing you cannot inspect — and the seller who could have opened the hood, and proved the engine sound, has already learned that no one was willing to pay him for the trouble, and stopped opening it. The record is how the hood opens. Until it is standard, the market clears where every market for an unverifiable good clears: at the bottom.

— Dispatches · Summit Cognitive

---