What the machine cannot say sorry for

An apology is one of the most demanding sentences a person can say, which is why it is so easy to fake. To mean it, you have to do several hard things at once: understand what went wrong, accept that you were the one who did it, recognize the claim the wronged party now has on you, and commit to setting it right. The words are the smallest part. A child can be made to recite them. What makes an apology real is everything standing behind the words — an agent who grasped the harm and can be held to the repair. Strip that away and you are left with the form of the thing, which is to say, with nothing.

This is worth holding onto as automated systems begin, fluently, to say sorry. A model can generate a flawless apology — better-phrased, often, than the human one it replaces, more measured, more sympathetic, free of the defensiveness a real person brings to admitting fault. It can name the harm, express regret, and promise improvement, all in well-formed sentences. And every word of it is hollow, not because the system is insincere, but because sincerity is not the kind of thing it can have. There is no one in there to whom the regret belongs. The apology is a description of an apology, performed by something that cannot be its author.

What the words presuppose

The hollowness is not a problem of feeling — we are not asking the machine to be sad. It is a problem of standing. An apology only does its work because it points at someone who can be held to it afterward. When a person apologizes, you are not merely receiving a pleasant sentence; you are receiving a name and a relationship. You now know who erred, and you know that if the harm continues, or the promised repair never comes, there is a party you can return to. The apology opens an account. It says: here is who is answerable, and here is what they have undertaken to do.

A system cannot open that account, because it cannot be on the other end of it. You cannot return to a model in a month and hold it to a commitment. It has no continuity of obligation, nothing that persists from the moment of the wrong to the moment of the reckoning, nothing that can be made worse off if it fails you. Punishment slides off it. Restitution means nothing to it. The entire machinery by which an apology becomes binding — the fact that the apologizing party has skin in the outcome — is simply absent. So when a system apologizes, the words arrive without the thing that gives words like these their force.

An apology is a promise about the future made by someone who can be found in it. A machine can make the promise and cannot be found.

This is the same vacancy I have written about under a different name. The decision nobody made is the decision that issues from a chain of systems with no identifiable author — and an apology nobody can mean is its sequel. When the harmful decision had no author, the apology for it has no one to be sorry. The two failures are the same failure seen from opposite ends: at the front, a consequence with no person behind it; at the back, a regret with no person inside it. In both cases the institution has produced the visible artifact of responsibility while quietly removing the responsible party.

Keep someone in the chain who can mean it

The conclusion is not that machines should be forbidden to express regret, or that automated decisions are illegitimate because no algorithm can feel remorse. That would be sentimental and beside the point. The conclusion is structural: behind any machine decision that can harm someone, the human chain has to remain identifiable, so that there is always a party who can do what the machine cannot — understand the wrong, accept the claim it creates, and be held to the fix. The apology can be drafted by a system. It cannot be owned by one. Ownership has to land on a person, or it lands nowhere.

This is the deeper reason a consequential record needs a signature. Who signs a decision is not a bureaucratic formality; it is the answer to the question an apology is supposed to answer — who is answerable here. A Decision Receipt that carries the identity of the responsible party does something the most eloquent automated apology cannot: it names a person who can be returned to. It preserves the chain back from the harm to a human who can be made to mean the repair. Without that, the apology is decoration on an empty room. With it, the words finally have somewhere to attach.

And the test of whether an apology is real has never been the quality of the prose. It is whether there is anyone, afterward, who is worse off if the wrong is not made right — anyone with something at stake in the keeping of the promise. That is the one thing automation removes and the one thing accountability requires. A machine can compose the most graceful regret ever written and still owe you nothing, because owing is a relationship and there is no one there to be in it. So when a system tells you it is sorry, the question to ask is not whether it means it. It cannot. The question is who, behind it, is in a position to — and whether the institution has kept that person identifiable, or quietly let the apology become the last place the responsibility was seen before it vanished.

— Dispatches · Summit Cognitive