DISPATCHES · Summit Cognitive

← All dispatches

LegitimacyThe CasebookJuly 27, 20265 min read

The exam the software watched

An accusation of cheating is a charge against a person's integrity, and when the accuser is software reading a student's eyes and room, the burden of disproving it lands on the accused — who cannot see, and often is never shown, what the machine claims it saw.

A student sits a final exam from a bedroom, laptop camera on, a piece of proctoring software watching. She is told to keep her face to the screen, her hands in view, her room quiet. She does. She also, in the ordinary way of a person thinking hard, glances up when she works a problem in her head, shifts in her chair, murmurs the steps under her breath. A dog barks in the next apartment. Two days later an email arrives: her exam has been flagged for suspected academic dishonesty. The software recorded gaze directed away from the screen, movement inconsistent with test-taking, and background audio during the session. A hearing will be scheduled. She has, the email says, the right to respond.

She reads it three times. She was not cheating, and she knows it — but knowing it is not the same as being able to show it, and she quickly discovers she can show almost nothing. She was never told, during the exam, that anything was wrong. She has no copy of what the camera saw, no timeline of the flags, no way to know whether the machine counted a glance at the ceiling as a glance at hidden notes. She is being asked to rebut an interpretation of her own body and her own room that she is not permitted to inspect. The charge is specific enough to threaten her degree and vague enough that she cannot get a grip on it. That gap — between how much is at stake and how little she can see — is the whole subject of this essay.

The machine that read your eyes

Begin with what actually happened, mechanically. A model watched a video feed and emitted signals: eyes off-axis for some number of seconds, motion above some threshold, sound above some level. Those signals were scored, and the score crossed a line, and the crossing became a flag. A flag is a statistical event. It is the software saying, in effect, this session looks less like the sessions we treat as clean and more like the sessions we treat as suspect. That is all it is. It is not a finding of dishonesty. It is a correlation with a threshold attached.

But watch what the flag becomes as it travels. By the time it reaches the student it has changed costume. It is no longer "an anomaly in the signal"; it is "suspected academic dishonesty," a phrase that names not a pattern but a wrong — a thing a person did, on purpose, against a rule. The accusation is now moral. And the burden has quietly inverted. The institution did not have to prove she cheated; the flag did the accusing, and she must now prove she did not. She must account for every second the machine found suspicious, using a record she does not hold, against a standard of normal behavior she was never shown. The one thing that would let her mount a defense — the specific evidence, the actual moments, the interpretation applied to them — is the one thing the process tends not to give her. She is defending against a shadow with a name.

An accusation owes due process

Here is the principle the situation forces into view. An integrity accusation is not an ordinary administrative act. It is quasi-judicial: it puts a person's standing on trial. To be found to have cheated is to be found to be a certain kind of person — dishonest, untrustworthy — and that finding attaches to a record that follows the student into every future room where their character is weighed. A decision that can do that to someone owes them the elements we have always demanded of a fair accusation, because we learned the hard way what happens without them. The accused must be told, specifically, what they are accused of. They must be able to see the evidence against them. They must be able to contest not just the conclusion but the interpretation — to say, of this exact moment, that is not what it means.

A charge against your integrity that you cannot see the evidence for is not a fair accusation; it is a verdict wearing the costume of one.

Automated proctoring, as it is often deployed, inverts each of these in turn. Instead of a specific charge, an opaque flag. Instead of the evidence in the accused's hands, a score they cannot audit. Instead of a presumption of innocence, a presumption baked into the framing: the machine flagged you, so explain yourself. And the signals themselves are thick with false meaning. Gaze that "leaves the screen" is how many people think; it is also how a person with a visual or attention-related disability works, how a person in a cramped or shared room sits, how anyone glances at a second monitor they were told they could use. Movement "inconsistent with test-taking" describes fidgeting, chronic pain, a child at the door, a body that does not hold still on command. Background audio is a household. None of these are cheating. All of them are, to a model trained to notice deviation, deviation. The student is not being measured against dishonesty. She is being measured against a statistical portrait of a compliant test-taker in a quiet, able, private room — and penalized for the distance between that portrait and her life.

Integrity pursued legitimately

None of this argues that exam integrity does not matter. It does. An institution that hands out credentials has a real interest in ensuring the credential means what it says, and cheating genuinely corrodes that. The aim is legitimate. The question this essay presses is narrower and harder: can a legitimate aim be pursued through an illegitimate instrument? An un-inspectable automated accusation is illegitimate not because its goal is wrong but because of what it withholds from the person it accuses. Authorization to run the software is not the same as the legitimacy to convict on its output — a distinction this series has made before, and one that matters most exactly here.

What would pursuing integrity legitimately look like? Three things, and none of them require abandoning the software. First, the flagged evidence is made inspectable and contestable — the specific clips, the specific timestamps, the specific interpretation the system applied — handed to the student as a record built for them to fight, the way a Decision Receipt exists to be argued against rather than merely believed. Second, a human being with real authority sits above the flag, empowered not to rubber-stamp it but to overrule it, and accountable for the judgment either way — because a person can be asked what they concluded and why, and an oracle cannot. Third, proportionality: an ambiguous signal cannot be permitted to carry a career-shaping consequence on its own. The weight of the outcome must match the strength of the evidence, and a glance at the ceiling is not strong evidence of anything. The difference between an institution defending the integrity of its exams and an institution convicting on an oracle is not the presence of the software. It is whether the accused can see what they are accused of, and be heard by someone who can be wrong and answer for it.

The student in this scenario may well be cleared. Many are. But being eventually cleared, after weeks of dread, by a process she could not see into, is not the same as having been treated fairly — and the ones who are not cleared, or who never appeal because they do not know they can, do not get an essay written about them. What they were owed was not a better model. It was the oldest thing an accusation has ever owed the accused: the evidence, in the open, and a fair chance to answer it.

The scenario above is illustrative — a composite drawn to show a pattern, not an account of any real person, company, or event.

— Dispatches · Summit Cognitive

Continue from here

Turn the argument into a practice.

Get new dispatches, assess how your organization handles consequential decisions, or explore Summit Cognitive.