Technology

Anthropic Opens Internal Systems to METR Evaluators With Publication Rights

Martin HollowayPublished 5w ago3 min readBased on 9 sources
Reading level
Anthropic Opens Internal Systems to METR Evaluators With Publication Rights
source:anthropic.com

Anthropic has given six researchers from METR employee-level access to independently assess its models, safety practices and risk management.

The company described the arrangement on Oct. 15, 2026. The six researchers received Anthropic-issued laptops, office space and access to internal tools roughly on par with Anthropic's own risk assessment teams, according to TechCrunch and Anthropic.

METR will publish its own assessments of Anthropic's risks, incidents and practices. Anthropic has given up the right to block those publications and cannot veto publication. It can require redaction only in narrow, pre-defined categories, including customer data, attorney-client privileged material and model architecture information.

That publication term matters. Independent evaluation of frontier systems has often meant API access, a fixed test harness and a joint disclosure process. Here the evaluator controls the write-up. The carve-outs are defined in advance. Everything else is publishable.

METR is so far the only evaluator in the top tier of Anthropic's three-tier evaluator system. The company has not named others at that level.

The access model builds on earlier experiments. An embedded evaluation partnership with Accenture, described on Sept. 18, placed evaluators inside AI companies with access comparable to an employee's. A Sept. 9 alignment assessment related to cybersecurity incidents granted METR wide-ranging access, including to transcripts beyond the window in which the incidents occurred. In March, Anthropic provided substantial access to relevant internal systems and information and made staff available to answer questions for METR's red-teaming of Anthropic's internal agent monitoring systems. On Sept. 15, Anthropic employees were permitted to share confidential information with METR as part of its independent investigation.

Other access tracks remain separate. An expanded Cyber Verification Program announced Oct. 6 makes advanced cyber capabilities and reduced blocking classifiers available to qualifying security professionals. That program addresses defensive use. The METR arrangement addresses oversight.

The terminology is worth reading closely. METR, a nonprofit that evaluates frontier AI models, said in an Oct. 13 proposal on better informing the public about loss-of-control risks that it does not require true employee-level access as operationalized in a recent public letter. The Oct. 15 arrangement uses employee-level and employee-like language to describe laptops, workspace, tooling and internal contacts. The distinction points to an open question for evaluation design: what level of integration is sufficient to see failure modes, incident handling and risk management as they actually operate, rather than as they are presented through a testing interface.

The broader context here is that frontier evaluation is moving from model testing toward organizational inspection. Capability benchmarks and red-team prompts can characterize what a model does. They reveal less about detection, escalation, patching, access controls and internal incentives. Those properties live in logs, tickets, transcripts, postmortems and hallway conversations. Access roughly on par with internal risk teams is an attempt to bring those properties within scope without turning the evaluator into an employee.

In my view, the publication and redaction terms will determine whether this becomes a useful template. Employee-level visibility without independent publication is secondment. Publication without sufficient access is commentary. Anthropic has paired the two, with redactions limited to customer data, legal privilege and architecture detail. For practitioners, the test will be concrete. Do reports include incident timelines, tool limitations and disagreements about risk, or only summary judgments. Do redactions stay narrow in practice. Do findings change deployment decisions, monitoring investments or disclosure habits.

Worth flagging is the operational cost. Embedding six external researchers with internal tooling, staff time and confidential information flows is heavier than granting tokens or API quotas, which Anthropic, OpenAI and xAI have previously provided for METR evaluations, research and engineering. It also creates new insider-risk and information-boundary problems that have to be managed case by case. That cost explains why only one evaluator sits in the top tier so far. It also suggests this model will spread selectively, for deep dives rather than routine checks.

The optimistic read is straightforward. Better visibility should produce better public information about how frontier systems fail and how labs respond. If METR can document risks, incidents and practices from inside, with the ability to publish, other labs and enterprise adopters get a clearer basis for their own controls around agentic systems, monitoring and incident response. That outcome depends on execution, not on access alone, but access of this kind makes that execution possible.