Anthropic and OpenAI seek influence over what outside auditors can publish

Reuters reported on October 13, citing two people familiar with the matter, that Anthropic and OpenAI are seeking substantial influence over what outside reviewers of their models can publish. According to the sources, both companies share one motive: the fear that audit findings will reveal trade secrets. There is also an awkward detail. Both labs very recently promised independent evaluators a free hand.

What is known so far

The report contains few facts, and they are worded cautiously. The negotiations the Reuters sources describe concern the terms on which third-party groups get access to the models and publish their results. The labs want a say in publication. The available part of the article does not specify how broad that say is. It names neither the specific evaluators nor the form the restrictions would take: text approval, the right to request edits, a delay, or an outright veto.

The original TNW article breaks off at a mention that last month Anthropic CEO Dario Amodei made some statement. The available fragment does not show what he said, so I will not recount it.

Where the conflict arises

An external evaluation of a model makes sense only when its creator does not control the final text. If a lab can soften the wording or delay publication, the report turns into an approved press release with someone else's logo on it.

The labs' fear is not unfounded, though. To test a model seriously, an evaluator needs deep interaction with it: non-standard access modes, early versions, details of how the system behaves. A report can reveal a lot about training methods, architectural decisions and weak spots. For companies competing for enterprise customers and carrying valuations in the hundreds of billions of dollars, that is sensitive information.

The industry already has a ready-made compromise model. Information security has long used coordinated vulnerability disclosure: the vendor gets a fixed window to fix the problem (90 days at Google Project Zero), after which the researcher publishes the result whether or not the vendor agrees. The labs could reasonably claim the right to check for factual errors and to review a report for leaks of real secrets. A veto over conclusions is another matter entirely.

What the report establishes

• Source: Reuters, two people familiar with the matter, both anonymous.
• Parties: Anthropic and OpenAI.
• Subject of the dispute: how much control the labs have over what outside reviewers publish.
• The labs' official argument: protecting trade secrets.
• The contradiction: recent public promises to give evaluators a free hand.
• Unknown: contract terms, the names of the evaluators, the approval mechanism.

How to read such promises in practice

For anyone who buys models, writes regulatory requirements or covers the industry, the takeaway is the same: the words "independent evaluation" mean nothing without the publication terms. There are several things to look at.

First: what exactly is approved before a report is released — the facts and secrets, or the wording of the conclusions. Second: whether there is a hard deadline after which the evaluator publishes the results without consent. Third: whether the report must state that part of the material has been withheld, and why. Fourth: who pays the evaluator. Funding from the company being evaluated is not a flaw in itself, but combined with control over the text it creates a conflict of interest.

Until both labs publicly disclose the terms, it is reasonable to read outside audit reports with allowance for possible editing. If Reuters is right, the nearest practical test is simple: will evaluators publicly state the terms on which they worked, and will those terms match what the companies promised earlier?