Procurement is where evidence becomes a deliverable

Organizations often buy an AI capability and assume assurance will follow. The contract names the service, price, availability, security terms, and support. Evaluation evidence is requested during diligence but not made a continuing deliverable. Change notices are vague. Incident cooperation is conditional. When the system later affects a consequential decision, the buyer discovers that accountability was never included in what it bought.

OMB Memorandum M-25-22 treats federal AI acquisition as a strategic governance function, addressing competition, performance, data, transparency, interoperability, and vendor lock-in. The memorandum applies to federal agencies, but the institutional lesson travels: procurement determines which evidence and control rights exist after the sales process ends.

A buyer does not receive what appeared in the demonstration. It receives the obligations written into the agreement and the technical affordances delivered with the service. If the vendor is not required to identify material model changes, preserve relevant records, support incident investigation, or provide export on exit, goodwill becomes the assurance architecture.

Buy the continuing claim

AI performance is not a static product attribute. Models, prompts, retrieval sources, policies, and infrastructure change. A procurement decision should therefore define the continuing claim the supplier must support: performance for which use, under which conditions, measured how, refreshed when, and disclosed with what uncertainty and limitations.

Acceptance testing is the beginning, not the end. The contract can establish triggers for reevaluation after material changes, new uses, serious incidents, or observed drift. It can distinguish routine maintenance from changes that require notice and customer choice. Without that distinction, the buyer has purchased a name while the operative system remains movable.

Assurance that is not a deliverable is a favor the vendor may stop providing.

Evidence rights should be proportionate. Not every buyer needs model weights or unrestricted access to proprietary data. Buyers do need enough information to govern their use: version identity, applicable documentation, evaluation summaries, known limitations, security and incident contacts, data-handling facts, and records necessary to explain local decisions.

The contract should say how those facts arrive. A portal that silently updates is not a reliable notice system. A generic status page may not identify the affected model. A yearly report may be too slow for an autonomous deployment. Format, recipient, timing, and effective date turn an abstract transparency promise into an operational deliverable.

Buy cooperation before the incident

Incidents reveal the limits of ordinary support terms. The deployer may need model-version confirmation, logs, preservation of evidence, technical personnel, regulatory coordination, and rapid containment. The provider may need local context to determine whether a pattern extends across customers. These exchanges should not be invented while the reporting clock runs.

The companion OMB memorandum M-25-21 links federal AI use to governance, inventory, risk management, and public trust. Together, the memoranda illustrate why acquisition and internal governance cannot be separated. Agencies can govern only the systems for which their agreements preserve sufficient knowledge and control.

Subprocessors and upstream models belong in the map. A prime vendor may depend on a provider with different retention, notice, and support terms. The buyer should know which dependencies are material, how changes flow down the chain, and whether the prime can obtain the evidence it promises. Contractual responsibility without upstream capability produces an indemnity, not an explanation.

Remedy for affected people should be supported too. Can the buyer retrieve the inputs and operative policy for a challenged decision? Can it correct data and rerun the process? Can it explain which party controls the relevant record? A vendor service that cannot support appeal may be inappropriate even when its average performance is attractive.

The exit is part of assurance

Lock-in is not only an economic concern. It can trap an institution in an assurance deficit. If switching providers destroys historical traceability, invalidates identifiers, or strands decision records, the buyer may continue using a system because it cannot preserve accountability through migration. Portability should cover evidence as well as data.

Exit terms should define export formats, metadata, version mappings, retention periods, deletion evidence, transition support, and continued incident cooperation for decisions made before termination. The end of the commercial relationship does not end appeals, investigations, or harms associated with earlier use.

Competition benefits from portable assurance. Suppliers can be compared on the quality of their evidence, change discipline, incident support, and interoperability rather than on claims buyers cannot test. Common fields and interfaces lower the cost of switching while rewarding providers that make their systems governable.

Procurement teams need technical, legal, operational, and domain participation early enough to change the requirement. Security review after vendor selection can identify a missing control but may have little leverage to obtain it. The request itself should state the decisions the system will support and the evidence the institution must retain.

Price evaluation should include the cost of governing the service. A lower license fee may conceal manual evidence collection, custom logging, duplicate testing, incident delay, or an expensive future migration. These costs are not peripheral compliance overhead. They are part of the total cost of operating the system in a way the institution can defend.

The buyer should test proposed evidence during the selection process. Ask finalists to map one sample decision from input through model and policy version to output, oversight, and correction. Ask them to demonstrate a change notice and an evidence export. Concrete exercises expose gaps that polished questionnaire responses are designed to smooth over.

A disciplined purchase asks a plain question: what will we need from this supplier when the system is wrong, changed, challenged, investigated, or replaced? The answer should appear in deliverables, interfaces, service levels, and exit terms. Procurement is not where accountability is outsourced. It is where the institution buys the means to remain accountable.

— Dispatches · Summit Cognitive