Production signals need a change gate

Production is the best source of cases the design team failed to imagine. It is also a noisy, adversarial, biased, and privacy-sensitive source. A customer may be mistaken. A staff workaround may violate policy. An attacker may manufacture a pattern. A temporary outage may create a cluster of strange behavior. Feeding every signal directly into an improvement loop can make the agent more responsive and less governed at the same time.

OpenAI describes production sessions, escalations, and quality signals revealing gaps after launch, with Codex investigating and suggesting updates that teams test against the deployed version before approving a controlled rollout. The crucial boundary is the approval path between observation and changed behavior. Production evidence should generate candidates, not self-authorize them.

The same signal can imply several repairs. A repeated transfer might mean the agent needs new knowledge, the policy is unclear, a tool is missing, or the human queue is rejecting valid work. Optimizing for fewer transfers without diagnosing the cause can teach the system to retain cases it should escalate.

Separate observation, proposal, and release

The observation record should state what happened, the affected segment, consequence, evidence quality, and competing explanations. The proposal should name the mechanism it changes and the predicted benefit and risk. The release gate should test the proposal against the target cases, regression corpus, protected groups, high-consequence scenarios, and operational limits.

Evaluation contamination deserves care. If production cases become tests and training examples without custody, teams may unknowingly tune on the same interactions used to claim improvement. Preserve case identity, access restrictions, transformations, and whether the example influenced a prompt, policy, grader, model, or benchmark.

Production should teach the system what to examine, not decide by itself what to become.

Rollout should remain bounded. A change can improve the observed cluster and degrade a different population. Start with a defined segment, compare the deployed baseline, monitor leading and consequence metrics, and retain the ability to reverse. The approval should expire if the system, data, or policy moves beyond the conditions that were tested.

Users and frontline staff are sources of evidence, not labels to be mined. Their corrections should remain attached to context and uncertainty. A complaint that the agent was wrong may identify a factual error, a legitimacy concern, or a policy disagreement. Treating every complaint as a model-quality datum erases the part of the institution the person is actually contesting.

Production examples need a controlled route into development. Remove or tokenize unnecessary personal data, preserve access restrictions, and record why the case is being retained. A difficult session may be valuable for diagnosis while still being inappropriate for broad prompt-development access. The improvement loop should narrow exposure as it moves evidence between operational, evaluation, and engineering environments.

Aggregate signals should lead to hypotheses, not direct edits. A rise in transfers may reflect model uncertainty, a broken tool, a policy revision, seasonal demand, or a newly staffed queue that makes escalation more useful. Before changing behavior, segment the signal and inspect representative cases. The proposed intervention should state which cause it addresses and which observations would disprove that explanation.

Rollout should preserve a contemporaneous comparison. Shadow the change, use a limited cohort, or maintain a stable holdout where consequence and fairness permit. Pre-launch replay shows how the proposal treats recorded cases; a live comparison shows how people and systems respond to it now. Both are needed because a better offline score can alter user behavior, queue load, or tool-state distributions in ways the replay cannot contain.

Close the loop with a release decision that can be audited independently of the team proposing the change. The decision should state what evidence was considered, which dissent remained, which population was authorized, and when monitoring will trigger reconsideration. Approval is not proof that the change is universally safe. It is a bounded institutional commitment made under identified evidence and uncertainty. Record rejected proposals too, so the same weak change does not return without new evidence.

Operate the boundary

The practical starting point is a named control for the observation-to-proposal-to-release change gate. Write the boundary in terms an operator can evaluate: the initiating principal, permitted purpose, affected resources, allowed consequences, escalation path, expiry condition, and evidence produced. A policy sentence is useful context; the enforced object and its observable state are what make the policy operational.

Test the boundary by injecting noisy, adversarial, conflicting, and segment-specific production signals and verifying that none can alter behavior without bounded evaluation and approval. Preserve the starting state, the agent's route, any intervention, the final effect, and the gaps in observation. Repeat the exercise after changing a model, tool, provider, policy, or data source. A control that passed once should not silently lend its assurance to a materially different system.

The leading signal is the share of proposed changes with documented evidence, regression results, rollout scope, reversal criteria, and an accountable approver. Pair it with a consequence measure so teams do not optimize the dashboard while weakening the outcome. Review both on a fixed cadence and after every material incident or migration. When the signal disappears, determine whether the risk disappeared or the instrumentation did.

The production change owner should own the decision to continue, narrow, pause, or expand the workflow. The owner needs authority over the control and access to its evidence; responsibility without either becomes ceremonial. Record the decision, the evidence cutoff, the residual uncertainty, and the next review date so the claim can age honestly.

— Dispatches · Summit Cognitive