Six evaluators from the nonprofit METR now have Anthropic hardware, office space and access to internal systems, according to the company's October 15 announcement. The outside researchers are being equipped to investigate from inside the company, not just test a model through its public interface.

That changes the questions an evaluator can ask. A test can reveal what an AI does under selected conditions. Internal access can help investigate how the developer notices problems and responds. But getting evidence out of a company requires a second kind of access: permission to publish.

Three levels, two different permissions

Anthropic describes a three-tier system:

  • Tier one: limited investigations; publication needs company approval.
  • Tier two: broader access; publication does not need approval, but specified information can be redacted.
  • Tier three: METR's arrangement adds access comparable to internal risk teams, with similar publication rights to tier two.

Customer information, privileged legal communications and model architecture details are generally excluded. Anthropic says it cannot veto METR's publication, but can require narrowly defined redactions. METR can publicly explain and criticize those omissions without revealing the protected material.

Those distinctions matter more than the umbrella label “independent evaluator.” A team that needs permission to publish an unfavorable finding has a different relationship with its subject from one that can publish and describe what was withheld. More access alone does not erase that difference.

A report should explain what it could not see

In its October 14 account of how it works with AI companies, METR makes transparency about limitations central to its approach. Readers should know the investigation's scope, access restrictions, important redactions and pressures that could affect its conclusions.

This is not a demand that every confidential record become public. It is a way to distinguish “we found no problem” from “we could not examine the relevant evidence.” Without that distinction, the same reassuring sentence could describe very different investigations.

METR says it has so far declined payments and donations from frontier AI companies, while accepting free model access it could afford to purchase. It also acknowledges a harder dependency: companies can withhold access, filter information and stop cooperating. Financial separation does not automatically provide investigative power.

METR's general policy and proposal are not the text of its specific Anthropic agreement. The public descriptions establish what the parties say they are doing, not a verified account of every contractual term or of how the arrangement works in practice.

Evidence, not a permission slip

METR's accompanying proposal confirms an ongoing in-depth Anthropic assessment. It also says these exercises are not audits, compliance checks or formal supervision. METR is not making deployment decisions, and the science is not mature enough for confident safety assurance.

That is why this announcement should not be read as a completed safety review. As with the limits of frontier-model evaluations, the value depends on the question investigated and the evidence exposed. Our account of tests versus real-world AI actions explains another part of that gap.

The useful outcome is not merely six outside researchers getting through Anthropic's door. It is a report that lets readers distinguish what the team observed, what it was prevented from examining and what the company required it to leave out. Insider access can make an investigation possible; those disclosures make its conclusions assessable from outside.