Detectability needs an interoperability test

A generator can prove that it wrote a marker into its original output. That does not prove the marker is detectable where the content will be consumed. Social platforms re-encode video. Messaging systems resize images. document tools flatten layers. Audio services normalize streams. Archive systems extract text. The distribution chain is an interoperability environment, not a transparent pipe.

Article 50 of the AI Act calls for machine-readable marking that is effective, interoperable, robust, and reliable as technically feasible, while the Commission's July guidelines provide current implementation guidance. Those adjectives are system claims. They cannot be demonstrated solely by inspecting the generator's output before another service touches it.

Interoperability testing should name both ends. Which producer writes the signal? Which independent detector reads it? Which format, version, codec, container, or protocol carries it? Which transformations are inside the supported boundary? A successful self-test is useful engineering evidence, but it is not evidence that an unrelated platform can recover the claim.

Test the distribution graph

Build a matrix from real routes: generator to editing tool, content-management system, CDN, social service, messaging channel, archive, and detector. Capture the original, each transformed artifact, and the detector result. The matrix should include both expected preservation and expected loss so operators do not mistake an unsupported route for a silent control failure.

False positives matter alongside misses. A detector that labels ordinary or historical material as synthetic can damage credibility and create incentives to ignore the signal. Test human-created controls, mixed-origin artifacts, benign edits, and adversarial modifications. Publish the operating characteristics for the supported population rather than presenting detectability as a binary property.

Interoperability is proved by another system recovering the claim, not by the producer remembering that it wrote one.

Version negotiation needs a failure behavior. If a detector encounters an unknown marker profile, it can report unknown rather than absent. If a platform strips a field, the receipt can distinguish transformation loss from generator omission. These states are operationally different and should not collapse into an unmarked label.

Standards and implementations will move. Maintain a compatibility register with producer version, detector version, format, transformation, observed result, and last test date. A passing result from an earlier codec or service pipeline should not silently authorize a changed route. The register is also how teams find the smallest repair when a platform update breaks recovery.

The strongest program includes external fixtures. Public test artifacts, reference detectors, and independently generated examples make it harder for one vendor's assumptions to define success. Confidential or abuse-sensitive tests may need restricted custody, but the organization should still preserve enough evidence for a reviewer to reproduce the declared boundary.

Make the obligation operational

Begin with the producer-detector-format-transformation combinations for which detectability is claimed. Express it as a control object rather than a policy summary: scope, triggering condition, applicable system or model version, permitted exception, effective time, evidence source, and the consequence when the control cannot establish compliance. This lets engineering, product, legal, and operations examine the same boundary without pretending their responsibilities are interchangeable.

The minimum receipt should retain original and transformed artifact hashes, marker profile, producer and detector versions, transformation route, expected result, observed result, and false-positive controls. Keep the record proportionate and protect confidential information, but make it possible to determine which rule, artifact, system version, and accountable decision governed the event. A folder of undated screenshots may show that work occurred; it rarely proves that the operative control held for the affected release.

Test the implementation by sending fixtures through independent editors, platforms, codecs, archives, and detectors, including unknown versions and mixed-origin content. Include ordinary cases, boundary cases, degraded dependencies, and known exceptions. Preserve the starting state, observed output, machine-readable evidence, user-visible result, and any human intervention. Re-run the test after changing a model, content pipeline, interface, standard, provider, or policy interpretation.

The interoperability owner with each distribution-channel owner should decide whether the evidence supports continued operation, a narrower scope, a compensating control, or a hold. The owner needs authority over the affected release and access to the evidence. Record unresolved interpretation separately from a technical defect so an engineering patch does not masquerade as a legal conclusion.

Monitor both presence and effectiveness. A marker can exist but be stripped downstream. A disclosure can render but arrive after exposure. A document can be submitted but refer to an obsolete model. Pair a control-presence measure with a consequence or comprehension test, give the claim a review date, and reopen it when a dependency changes.

Maintain a dependency register for the control. Model endpoints, editing pipelines, content formats, user interfaces, identity services, submission portals, vendors, and external standards can change the evidence without changing the policy text. Name which changes invalidate the last test and which monitoring signal proves that the dependency remains inside the reviewed state.

Exercise the exception path as carefully as the ordinary path. Record who can invoke it, which facts they must supply, how long it lasts, what capability or distribution is reduced, and which compensating evidence remains. An exception without expiry and re-entry criteria becomes a second operating model that can silently outlive the reason it was approved.

Keep public and executive claims no broader than the tested boundary. Say which systems, releases, formats, routes, and dates the evidence covers, and identify material exclusions. When a control fails or a dependency moves, update the claim and the remediation record together. A transparent limitation protects more credibility than a universal statement built from a narrow passing test.

— Dispatches · Summit Cognitive