Rehearse the reconstruction

You built the logging. You know you did — the events fire, the tables fill, the retention policy is set, the dashboard is green. Somewhere in a design review a year ago, someone asked whether the system was auditable, and the honest answer was yes, we log everything. The box got checked and everyone moved on. What almost nobody does, in the interval between shipping that logging and the day it is finally called on, is the one thing that would tell them whether any of it works: sit down and actually try to reconstruct a real decision from what was retained. Not confirm the events exist. Reconstruct the decision — assemble, from the record alone, the full answer to the question someone will eventually ask under pressure. That answer is the only deliverable the logging was ever for, and it is the thing that has never once been produced.

So do it deliberately, on your own schedule, before anyone forces you to. Pick a decision your system made — a real one, with a real subject — and treat every gap you hit while reassembling it as a defect to be filed and fixed, exactly as you would a failing test. This is the directive: rehearse the reconstruction. Not because the record is likely to be demanded tomorrow, but because a record is only as good as the reconstruction it can support, and you have no idea how good yours is until you have tried to build one.

The drill you never ran

The question you will one day be asked has a specific and unforgiving shape: show me exactly why this decision was made — on what inputs, under what policy, by what version, at what moment. Check-the-box logging answers a different, easier question — did the system record that a decision occurred — and the two feel close enough that teams conflate them for years. They are not close. Recording that a decision happened is bookkeeping. Reconstructing why it happened is accountability, and the distance between them is exactly the distance you discover the first time you try to cross it under duress.

And the first time is almost always under duress. A regulator opens an inquiry, a lawyer files, a customer who was denied something goes public, and now someone on your team is asked to reconstruct a decision from eighteen months ago against a clock. That is when the gaps surface, all at once, in the worst possible light. The inputs the decision actually rested on were fetched live and never persisted, so you have the score but not what produced it. The policy version is unknown, because the rules were ambient config that has since moved and nothing recorded which cutoff was in force that day. The timestamps do not line up — the decision event, the model call, and the policy read are in three systems with three clocks and they cannot be reconciled to a single instant. Each of these is survivable as a finding in a drill. Each is a catastrophe as a discovery in a deposition. The record you were so sure of turns out to be a record you only thought you had.

Rehearsal turns failures into findings

The entire value of rehearsal is that it moves the moment of discovery. The gap between what you retained and what the question requires exists whether or not you look for it; rehearsal simply decides when you find out. Find it in a drill and it is a cheap finding — a ticket, a schema change, an added field, closed in a sprint while nothing is at stake. Find it under legal or regulatory pressure and it is an expensive failure — a decision you cannot defend, a subject you cannot answer, an institution that has to say, in effect, we do not know why we did this to you. Same defect, two prices, and the only variable is whether you rehearsed.

This is also the only honest test of whether your accountability infrastructure works. Log volume does not prove it works. Green dashboards do not prove it works. Passing a schema validation does not prove it works. The only thing that proves a record can answer the question is answering the question — reconstructing a real past decision, cold, from the record, and seeing whether the answer is complete, correct, and standing on its own without the live system to lean on. Everything short of that is a proxy, and every proxy is a way of believing you are covered without ever checking. A record is worth precisely the reconstruction it can actually support, and until you have attempted one you are holding a promise, not a proof.

A record you have never reconstructed is a fire extinguisher you have never taken off the wall — you will find out whether it works at the worst possible moment.

Run it like a fire drill

Treat reconstruction the way a serious operation treats fire and disaster recovery: as a scheduled, recurring drill, not a hope. Put it on a cadence. Pick real decisions — and weight deliberately toward the consequential and the likely-to-be-contested, and toward the hard edge cases where the logic branched in an unusual way, because those are the ones that will actually be litigated and those are where the record most often falls short. Then reconstruct them cold. Use only what was persisted: no live database, no current config, no engineer narrating from memory how it worked at the time. Assemble the inputs, the policy in force, the version, the operating point, the timestamped sequence, the outcome — and the moment you reach for something the record does not contain, stop and log the gap. Then close it. Every drill should end with a shorter list of defects than it started with.

Measure readiness the right way. Not by how much you log — volume is often the enemy of a usable record, not evidence of one — but by how many real decisions you can successfully reconstruct, cold, on demand. That is the number that means something. A team that can reconstruct nine of ten sampled decisions completely is in a defensible position; a team with a hundred times the log volume and no successful reconstruction is not, however green its dashboards look.

Two honest caveats keep this from becoming a fantasy. First, you cannot rehearse every decision, and you should not pretend to — sample, and let the sample lean toward what is consequential and contestable, so that when the demand comes it lands on terrain you have already walked. Second, the drill tests more than data presence. It tests the process of reconstruction: who does it, with which tools, how fast, and whether that person can produce a defensible answer under a clock when the request is real. A record that technically contains everything but takes three weeks and a departed engineer's tribal knowledge to assemble has failed the drill as surely as one with missing fields. You are rehearsing the answer and the act of answering, together — because when the first question comes after something goes wrong, both have to be ready, and the only way to know they are is to have done it before.

— Dispatches · Summit Cognitive