DISPATCHES · Summit Cognitive

← All dispatches

AccountabilityThe CasebookJuly 27, 20265 min read

The loan the model approved

We ask what a system owes the person it turns down, and forget the person it says yes to — but an approval can be the more damaging decision, and a model optimized to approve has every incentive not to keep the record that would show it should not have.

Picture an applicant who needs a few thousand dollars to close a gap between one hard month and the next. The form takes four minutes. An automated underwriting system reads the thin file — an income that arrives in irregular bursts, obligations that already crowd the calendar, a history of near-misses — and it approves. Not a modest amount, either, but a line larger than the numbers can bear, at a rate priced for exactly this kind of borrower, on terms that will compound faster than the income can catch. The applicant feels a small flush of relief. Approved. Someone looked, and the answer was yes. What the moment conceals is that the yes is the injury — that a denial would have been the kinder decision, and the system chose the one that pays.

We have trained ourselves to imagine algorithmic harm as a refusal. The wronged party, in our standard picture, is the person a model rejects — filtered out, scored down, turned away at a door they cannot see the mechanism of. That picture is not wrong. But it is partial, and its partiality has a cost, because it leaves an entire class of damage unattended. A decision to approve is still a decision. It can be the wrong one. And when it is wrong, it can hurt more, and more durably, than any denial — while attracting almost none of the scrutiny a denial invites.

The harm that says yes

Consider why the wrongful approval hides so well. At the instant it happens, no one in the room is unhappy. The borrower wanted the money and has it. The lender has booked a receivable at an attractive yield and is, in the most literal sense, paid to have said yes. There is no aggrieved party standing at the counter, no complaint filed, no obvious moment where someone can point and say that was the harm. The damage is deferred. It arrives later, in arrears and fees and the slow arithmetic of a debt that was never affordable, by which time the decision that caused it is months gone and buried under everything that came after.

Contrast the denial. A denial produces its wronged party immediately and legibly. Someone is told no, feels the sting of it, and — increasingly, and rightly — has some standing to ask why. We have built a whole apparatus of adverse-action reasoning around the no: an obligation to explain, a right to contest, a record of grounds. The yes has no equivalent. The approved borrower is, at first, pleased; pleasure does not file grievances. So the decision that most needed a second look is precisely the one nobody looks at, because at the moment of looking there is nothing yet to see.

This is the asymmetry the accountability conversation has mostly missed. It fixates on denials because denials come pre-supplied with a plaintiff. But the question that accountability actually asks is not who is upset — it is should this decision have been made? And that question is agnostic to direction. It applies to the yes exactly as much as to the no. A system can wrong you by opening a door as surely as by closing one, and the fact that you walked through it gladly does not settle whether the door should have been opened.

The incentive to approve

There is a deeper reason the wrongful approval goes unrecorded, and it is not oversight. It is incentive. A system built to maximize approvals, or volume, or booked yield, is optimizing for something that diverges — at exactly the margin that matters — from the borrower's interest. For most of the range the two align: approving a creditworthy applicant serves everyone. It is at the edge, where affordability is genuinely in doubt, that the objectives split. There, the borrower is best served by a no, and the system is best served by a yes. The approval that harms is not a malfunction of such a system. It is the system working as designed, in the region where its design and the borrower's welfare point in opposite directions.

Now ask what record such a system has any reason to keep. A decision-maker preserves, by default, the evidence that protects it. A denial exposes the decider to challenge, so the grounds for denial get documented. But an approval, in the moment, looks like a gift and a sale at once — it draws no challenge and produces revenue. The incentive to build a durable, reconstructable account of why we said yes, and what we understood about whether this person could carry it is close to nil. The very decision most in need of a record is the one the decider has the least reason to make legible. Accountability that only functions when the decider volunteers the record will never reach the wrongful yes, because the wrongful yes is defined by the decider's interest in leaving it in the dark.

We built elaborate rights for the person a model rejects and almost none for the person it approves into ruin — as if only a no could be a wrong.

What the account owes even the approved

The remedy is not to make systems approve less, or to second-guess every yes as if generosity were the crime. It is to insist that an approval, like a denial, leave behind a record that can be reopened. The question a decision owes its subject is answerable only if the decision preserved the material to answer it: what the system knew about this person's capacity to bear the obligation, what it was actually optimizing for when it weighed that capacity against the yield, and why — given both — it landed on yes. Not a marketing summary of responsible lending. The reconstructable state of the decision itself, kept because the decision was consequential, not because a regulator asked.

This is what a Decision Receipt is for, and it is worth being precise about the claim. The point is not that the record proves the approval was wrong. The point is that the record lets the question be asked at all — later, by the person the decision landed on, or by anyone who acquires standing to contest it — and answered against the evidence that was actually in front of the system, not against a reconstruction assembled after the fact by the party with the most to lose. An approval that can be replayed and interrogated is one whose author can be found. An approval that vanishes into a booked balance is a decision no one need ever answer for, made by a system whose incentive was never to let it be found.

So the discipline the yes demands is the discipline we already grant the no, extended to the direction we forgot. Scrutinize the approval as hard as the rejection. Require of the decision that opens a door the same reconstructable account we require of the one that closes it. Treat the borrower who was approved into a debt they could not carry as a wronged party with standing, not a satisfied customer who got what they asked for. Accountability that only interrogates denials is accountability that has agreed, in advance, to look away from half of what a system does — and, given the incentives, very likely the more damaging half.

The scenario above is illustrative — a composite drawn to show a pattern, not an account of any real person, company, or event.

— Dispatches · Summit Cognitive

Continue from here

Turn the argument into a practice.

Get new dispatches, assess how your organization handles consequential decisions, or explore Summit Cognitive.