Take nobody's word for it

The Royal Society took as its motto a fragment of Horace: Nullius in verba. The fellows chose it soon after the Society's founding in 1660, and it was fixed in the charter granted in 1662. It is usually rendered "take nobody's word for it," and the full Horatian line it is clipped from — nullius addictus iurare in verba magistri, "not bound to swear by any master's words" — makes the defiance plainer still. The motto was a refusal: questions about the natural world would no longer be settled by the authority of an ancient text or a learned man, but by appeal to facts that could be produced and checked. It is hard, now, to feel how radical that was. For most of history, knowing meant trusting the right person. The Society proposed to replace trust with a procedure.

The procedure had a centre, and the centre was a machine. Robert Boyle, a founding fellow, had built an air-pump — an expensive, temperamental device for evacuating a glass vessel — and with it ran the first careful experiments on the behaviour of air at reduced pressure, published in 1660. What matters for our purposes is not the physics but the social technology Boyle invented around it. An experimental result, he argued, became a matter of fact not because he asserted it but because it was produced in front of credible witnesses and could be produced again. The experiments were described in such exhaustive detail that a reader could, in principle, build the apparatus and reproduce the result for himself. Boyle's reasoning was almost statistical: the chance that many independent, reliable observers are all wrong about the same thing is lower than the chance that one authority is wrong.

The historians Steven Shapin and Simon Schaffer gave the deepest move a name: virtual witnessing. Boyle could not bring all of Europe into his room, so he wrote the experiment up with enough granular, replicable specificity that a distant reader became, in effect, a witness — able to follow the procedure step by step and judge for himself whether the conclusion followed. The written account was engineered to stand in for being there. A claim, on this model, is not a thing you accept on the speaker's say-so. It is a thing that arrives carrying enough of its own method that you can re-run it and find out.

The motto was not "trust the experts." It was the opposite: bring the method, not your authority, and let the result be produced again by someone who doubts you.

The standard was reproducibility, not eloquence

Notice what this standard refuses. It refuses to let fluency, eminence, or confidence settle anything. A result delivered by the most distinguished fellow in the most persuasive prose counted for nothing if it could not be reproduced; a result from an unknown counted fully if it could. The whole apparatus was designed to make the credibility of a claim a property of the claim's method rather than of the claimant's standing. That is the inheritance the modern world calls the scientific method, and reproducibility — the demand that a finding travel with enough of its procedure to be independently re-run — has been its load-bearing pillar for three and a half centuries.

Now look at how we handle machine claims. A model returns a score, a classification, an answer, often in fluent and confident language. And we have, almost without noticing, reverted to the pre-1660 epistemology the Royal Society was built to overthrow. We accept the output on the strength of how it sounds and who deployed it. We cannot re-run it: the model is nondeterministic, or the inputs were not preserved, or the version has since changed, or the surrounding context that shaped the answer is gone. The result does not arrive carrying its method. It arrives as an assertion from an authority we are asked to trust — which is precisely the thing the motto forbids.

Bring the method, not the authority

The fix is not to demand that every model become a peer-reviewed laboratory. It is to recover what virtual witnessing actually required and build it into how consequential machine outputs are recorded. A finding is reproducible when a skeptic who was not present can take the preserved account — the inputs as they stood, the version in force, the procedure followed — and run it again to the same result. That is a precise, checkable property, and it is the difference between a claim and a matter of fact. A decision that can be replayed against what was actually known has, in Boyle's exact sense, brought its witnesses with it. A decision that can only be re-described after the fact has brought nothing but the master's word.

This is also why a model's explanation of itself is worth so much less than it appears. When you ask a system to account for its output, you do not get the procedure re-run; you get a new, fluent statement generated under no obligation to match what actually drove the original result. That is testimony, and unverifiable testimony at that — the very thing Nullius in verba was raised against. The Royal Society's answer to "how do I know you're right?" was never "here is my eloquent account of my reasoning." It was "here is the method; produce the result yourself." A machine claim that cannot be re-run has refused that answer, however articulate its self-description.

We tend to think of the scientific method as a discovery about nature. It is at least as much a discovery about trust — a hard-won technique for settling disputes among people who do not, and should not, simply believe each other. That technique was built for a world drowning in confident assertions backed by authority and not by anything you could check. We are about to live in such a world again, at a volume the fellows could not have imagined, saturated with fluent machine claims that ask to be taken on their word. The three-and-a-half-century-old answer is not obsolete. It is the thing we are most tempted to skip and can least afford to: take nobody's word for it — not the master's, and not the machine's. Bring the method, and let the result be produced again.

— Dispatches · Summit Cognitive

---