Outside AI tester says Anthropic blacked out key evidence in its report

An outside group that tests AI systems says the public version of its report is missing much of its best evidence. The group is METR. It says Anthropic, the company that makes the Claude AI assistant, redacted, or blacked out, a lot of the material METR called the most informative.

What happened

METR says it submitted a full report of about 100 pages on its assessment to Anthropic. For the public version, Anthropic redacted much of the evidence METR considered most informative. METR says the result is that readers "must rely on our judgment more than we consider ideal." The statement was posted on METR's account on X.

Why it matters

Outside testers exist so the public does not have to take a company's word that its AI is safe. That only works if people can see what the testers saw. With key evidence removed, readers get conclusions without the proof behind them. They have to trust METR, and trust the redaction choices, instead of checking for themselves.

This matters most to anyone who relies on AI tools for work, money or family decisions.

What is not known yet

• What exactly was redacted, and how much of the report it covers.
• Why Anthropic chose to redact it.
• Whether Anthropic will respond or release more of the report.
• What METR's overall findings were.

The material available here is only METR's short statement. It does not include Anthropic's side or the report itself.

What happens next

Watch for a reply from Anthropic and for any update from METR. A fuller public version would let readers judge the evidence directly.