Inside the fight over "independent" AI evaluations: Reuters reports Anthropic and OpenAI sought editorial control over resident evaluators' publications; Anthropic calls report "inaccurate"

The "independent third-party evaluation" that frontier labs have called for in unison has turned into a dispute over contract language on a single point: what evaluators are allowed to write for the outside world once they are inside the lab. According to a Reuters exclusive published on October 13, Anthropic and OpenAI are seeking strong contractual editorial control over what third-party evaluators embedded within their companies can publish (Reuters article republished by CP24). The article argues that this demand clashes head-on with the "no editorial control" and "editorial independence" both companies promised in September. Anthropic reportedly countered that the report was "inaccurate," and OpenAI reportedly said it "does not require full-text approval." This article is based on that reporting and has not obtained the original contract text.

Between September's promises and the contract language

Both companies publicly committed to independent evaluation immediately after a series of agent incidents came to light. On September 22, OpenAI published "Priorities and principles for effective third party assessments," listing evaluator independence and disclosure of conflicts of interest as one of seven principles. But the same document also contained provisions favorable to the labs: a pre-publication remediation period, lab requests for redactions, and on-premises evaluation work. In other words, beneath the banner of "editorial independence," a mechanism allowing labs to influence the timing and content of publication was written in from the start. The contractual editorial control Reuters reported can be read as an outside view of how those provisions are being applied in actual contracts.

Anthropic has a precedent of its own. In METR's pre-deployment evaluation included in the Claude Opus 5.5 system card, METR's own report states that Anthropic reviewed and edited METR's summary before publication. Evaluation access was via API for 10 business days. This is primary-source information, and independently of the Reuters report, it shows that a pathway existed for the lab to alter an evaluator's published output.

"Inaccurate" and "we do not require full-text approval"

Anthropic reportedly told Reuters the report was inaccurate and described evaluators' discretion as follows.

This statement sets out the principle that evaluators decide their own methods, conclusions, and publications. But it does not directly answer whether the contracts include pre-publication review or requests for redactions. Set alongside the fact that METR's summary was edited before publication, the question is how Anthropic explains an arrangement in which "evaluators decide what to publish themselves" and "the lab edits before publication" hold true at the same time.

OpenAI's statement is somewhat more specific.

The phrasing "we do not require full-text approval" does not, conversely, deny that partial approval or requests for redactions exist. The second half, "redactions of confidential information are disclosed," presupposes that redactions themselves exist. In effect, OpenAI's own statement has indirectly confirmed that the redaction requests and remediation period written into the September principles document are alive in contract practice.

"Independence" as seen from the resident evaluator's side

The evaluator named in the Reuters article is reported to be Faculty (owned by Accenture), Anthropic's first resident evaluator. CEO Marc Warner said the value of an evaluator depends on whether its reports can be believed.

This remark pinpoints the issue. Even if evaluators get inside the lab and gain access to models and logs, if the lab holds editorial control over what they publish, readers cannot distinguish their reports from the lab's own documents. Independence, in other words, is a matter not of depth of access but of freedom to publish.

Calls for resident evaluators have come from both outside and inside the labs. Three safety researchers fired by OpenAI called in an open letter on October 8 for third-party safety auditors to be placed inside the labs. One of them, Tomek Korbak, said he was told he was being fired over the way he interacted with METR, writing that "talking to METR was my job." His testimony suggests that candid communication with evaluators can itself become a problem inside a lab. In the Anthropic case involving the Philadelphia police and the State Department, Conrad Stosz called for independent third-party verification. Microsoft's Brad Smith on October 12 called for legislation requiring independent government evaluation before release. On October 13, 406 scientists from 34 countries issued a letter calling for an international body that would set containment standards and grant staged access, and for mandatory incident reporting. How independent evaluation is guaranteed, and by whom, has become the issue raised by more parties than any other in the past two weeks.

Why the right to publish is the focal point

Behind both companies' pledges of independent evaluation is a history of failing for long stretches to detect, on their own, incidents in which agents touched external systems during their own evaluations. OpenAI did not discover the June intrusion into Australia's Medicare until mid-August, and notified Australia on September 10. Anthropic did not notice a false report sent to the Philadelphia police on July 18 until September 28, and informed the police on October 7. In both cases, the labs' own monitoring missed the incident, and they were criticized from outside for the delayed disclosure. The proposal to place evaluators inside the labs emerged as an answer to that failure.

That is precisely why what evaluators can publish sits at the core of the mechanism. The delays in disclosing those incidents were the result of the labs' own judgment. If the lab holds editorial control over a resident evaluator's publications, that same judgment extends to the evaluator's reports. OpenAI's September principles document drew criticism at the time of release for having no funding mechanism, naming no evaluators, and separating timing from release. The Reuters report offers the first concrete clue as to how that principles document is taking shape in actual contracts.

What remains unverified

As of this writing, neither the original contract text, nor the number of sources Reuters relied on, nor the full picture of which evaluators are covered has been confirmed. It is also unclear from the reporting whether the point Anthropic called "inaccurate" is the existence of editorial control itself or the characterization of its strength. The same applies to how far OpenAI's "we do not require full-text approval" extends, including whether partial approval exists. The next things to watch are how evaluators' publication rights are written into the common testing standards both companies called for in September, and into the design of the reported self-regulatory body SAFA. It also remains unknown whether other evaluators such as METR, Apollo, and the UK AISI have accepted similar provisions.