METR's Claude Opus 5.5 safety assessment relied on undisclosed sources
In brief
- METR evaluated Claude Opus 5.5 using restricted access and partly undisclosed sources
- External testing found no sustained AI development pace doubling versus Anthropic's internal assessments
- METR's report was marked experimental and preliminary with non-public evidence
- Anthropic also relied on undisclosed information in its own model evaluations
Restricted Access and Preliminary Findings
METR's external testing did not identify a sustained doubling in the pace of AI development compared to Anthropic's own internal assessments. The team operated with restricted access during their evaluation, a constraint that limited the scope of what they could examine.
The nonprofit's report was described as "highly experimental and preliminary." The evidence behind those conclusions can't be shared publicly, according to METR's own disclosure. This opacity complicates independent verification of the findings.
Anthropic's Own Evaluations
Anthropic itself utilized another undisclosed source of information in its evaluations. The company released the Claude Opus 5.5 System Card on September 22, 2026, documenting improvements across several domains.
Claude Opus 5.5 shows improvements in agentic coding capabilities, computer usage, and performance on long-horizon professional tasks. These advances form part of Anthropic's broader capability roadmap, though the underlying evaluation methodology also relies on sources the company has not disclosed.
The reliance on non-public information by both evaluator and vendor underscores a structural challenge in AI safety assessment: the tension between detailed technical evaluation and public accountability.
