Test the whole decision

Open your test suite and look at what it actually exercises. You will find, almost certainly, that it exercises the model. There is a held-out set, and there are numbers computed against it — accuracy, AUC, an F1 you can quote to three decimals — and a gate that blocks the merge if any of them slips. It is disciplined work, and it is real. But run your eye down the pipeline from that model output to the action your system finally takes, and count how many of the steps in between have a test that names them. The threshold that turns a score into a decision. The business rule layered on top of the threshold. The fallback when a feature comes back null. The default when the scoring service times out. In most systems the honest count is zero. You have tested the estimator to four decimal places and shipped the decision untested.

The directive is one sentence: test the decision, end to end, not the model in isolation. The model is a component. The decision is the whole path — score, then threshold, then the rules, then the fallbacks, then the action that lands on a real person — and the decision is the object that has consequences and therefore the object that owes a test. A component can be excellent and the assembly still wrong. Every engineer knows this about every other system they build; we suspend the knowledge the moment a model is in the loop, because the model is the interesting part and the pipeline around it feels like plumbing. But the plumbing is where the water goes.

The model is tested, the decision is not

Consider where a real error is actually born. It is rarely a misclassification in the strict sense — the model doing its job badly on inputs it was fairly given. It is far more often a seam. The score is sound, but the threshold that consumes it was set for a different population, or was copied from a notebook and never revisited, and so a correct probability produces a wrong action. Or a feature is missing and the fallback quietly substitutes a zero, which the model reads as a strong signal rather than an absent one, and the decision inverts. Or the scoring call times out under load and a default fires — a default someone chose in a hurry two years ago, that nobody has looked at since — and for the duration of the incident every decision is really that default wearing the model's authority. None of these are model errors. Every one of them is a decision error, and not one of them is visible to a metric computed on the model alone.

This is the gap the four-decimal-place discipline hides. The precision of your model metrics is not evidence that the decision is sound; it is evidence that the tested part is sound, which was never the part in doubt. The untested part is the threshold, the rule, the fallback, the default — the layer that translates an estimate into an act — and it is untested precisely because it does not look like machine learning. It looks like configuration and glue. So it escapes the test set, escapes the review, escapes the dashboard, and lives in production entirely on the strength of having once seemed obvious. Most of the decision, by line count and by risk, was never on any test set at all.

Build a test surface for the decision

The remedy is to give the decision its own test surface, distinct from the model's, and to hold it to the same standard you already hold the model to. Three kinds of test, concretely. First, golden end-to-end cases: fixed inputs run through the full path — score, threshold, rules, action — with the expected decision asserted at the end, not the expected score in the middle. A golden case is a decision you have reasoned about carefully and are willing to pin, so that when someone changes a threshold or edits a rule six months from now, the case that used to resolve one way and now resolves another lights up red and makes them look at what they moved.

Second, failure-mode tests that assert the decision degrades safely. Feed the pipeline a missing feature and assert it does not silently coerce absence into a confident signal. Simulate the timeout and assert the default is the conservative action you would actually defend, not whatever the code falls through to. Degrade an input and assert the decision either holds or refuses rather than confidently proceeding on a corrupted basis. The point of these tests is not to prove the failure never happens — it will — but to prove that when it does, the decision fails in the direction you chose, on purpose, and can say so.

Third, replay of real historical decisions. Take a body of decisions the system actually made, and re-run the current pipeline against the inputs those decisions rested on. You are not checking the model here; you are watching for drift in the pipeline — a threshold that has crept, a rule that was amended, a fallback whose behavior changed under a dependency upgrade — that a held-out model metric would never surface, because the model is unchanged and only the machinery around it moved. Replay against what was actually known is the one test that catches the decision drifting while the model sits still.

A model that passes every test inside a decision that was never tested is a well-inspected part bolted into an uninspected machine.

The objections, answered

The first objection is that the model is the hard part, so the model is where the testing effort belongs. This has the reasoning exactly backwards. The model is the hard part, which is precisely why it is the tested part — hard problems attract attention and budget. The pipeline is the easy part, which is why it is the untested part, and the untested part is where systems fail. Difficulty is not risk. A threshold is trivial to write and catastrophic to get wrong, and its triviality is the whole reason nobody guards it. Effort should follow exposure, not intellectual interest, and the exposure is in the glue.

The second objection is that golden cases are hard to build — that you cannot enumerate every path, that fixing an expected decision requires judgment you would rather not commit to a file. This is true, and it is the point. The difficulty of stating what the right decision is on a hard case is not a reason to skip the test; it is the reason the test is worth having. If you cannot say what your system should decide on a case, you do not have a specification, you have a model and a hope. You need only a handful — the boundary cases, the ones near the threshold, the ones that exercise each fallback — not a golden case for every input. A small set chosen for where the seams are will catch more real regressions than a million held-out rows that only ever touch the model.

The third objection is non-determinism: some pipelines will not reproduce a decision byte for byte, so end-to-end assertions feel impossible. Sometimes true, usually overstated. Much of what reads as non-determinism is unpinned inputs and floating config, and pinning them is the same discipline that makes a decision reconstructable in the first place. Where genuine stochasticity remains, assert on the decision's properties rather than its exact value — that it stays on the safe side of the line, that it falls in the defensible band, that it does not invert. A decision you can only assert loosely is still a decision you can assert; a decision you assert nothing about is one you are shipping on faith.

Which returns this to the series. A decision you cannot test end to end is a decision you cannot stand behind, because you have only ever verified a component of it — and “the model passed eval” is not “the decision is sound.” The eval speaks for the estimator; the decision was made by the whole pipeline, and the whole pipeline is what a Decision Receipt has to answer for and what a replay has to reproduce. Test the part and you can vouch for the part. Test the decision and you can vouch for the act. Only one of those is the thing you will be asked about.

— Dispatches · Summit Cognitive