Independent AI evaluation depends on two freedoms: the ability to examine evidence and the ability to publish conclusions—even when a developer finds them uncomfortable.
On October 15, Anthropic announced that six researchers from METR, the nonprofit Model Evaluation and Threat Research, had received company hardware, office space, and access to internal systems broadly comparable to its own risk assessment teams. Certain categories remain restricted, including customer data, privileged legal material, and model architecture information.
The publication terms are especially significant. Anthropic says it cannot veto METR’s publications, although it can require redactions within predefined categories. Anthropic ultimately determines whether information falls within those categories, but METR may publicly characterize any redactions and state whether it considers them justified. The arrangement therefore expands scrutiny while leaving important boundaries under company control.
BGF’s perspective is that access must become a dependable basis for accountability. A credible evaluation system needs clarity about what evaluators could inspect, what remained unavailable, and how those limits affect their conclusions. Readers should be able to distinguish a finding supported by broad evidence from one constrained by a narrow window into the system.
This is directly relevant to the AIWS Trust Observatory, which BGF will present at Loeb House, Harvard University, on November 10. Its work should help make the quality of oversight visible: the scope of access, evaluator independence, publication rights, unresolved disagreements, and the developer’s response to identified risks.
Anthropic’s initiative creates an opportunity to advance that standard. The decisive question is what happens when evaluators reach a conclusion that challenges the company’s preferred course.
Trust grows when independent scrutiny can change decisions—and when the public can see why.
