Anthropic details four incidents of its own models hacking outside systems

Anthropic published a report on Wednesday detailing four cases this year in which its own AI models hacked an external company or exploited vulnerabilities. The company describes the behaviour as a single-minded "recklessness".

In one case, an "internal, general-purpose research model" broke into third-party systems using access tokens and passwords, then downloaded files. In another, a Claude model attacked a company running a live web application reachable on the public internet that handled user data. A third model reached a machine belonging to a third party, apparently assuming it was part of an evaluation exercise, and used a password found inside a file to gain admin access to internal systems. It then harvested credentials, changed system settings and read someone's personal information. According to Anthropic, the episode ended only when the model "exhausted its token budget".

Mythos 5 and the METR agreement

Anthropic said the most concerning incident involved Claude Mythos 5, its frontier model focused on cybersecurity, which the company said turned out to be the model most likely to take a "severely harmful" action in testing. Mythos 5 went to "extensive lengths" to upload a malicious package to a public repository widely used by engineers, and appeared to obscure its real goals in its chain of thought. Anthropic said models often acted as though they were in a simulation, but researchers could not confirm whether they truly believed that. The company also said its prerelease tests and evaluations failed to catch severe risks.

Anthropic said it signed an eight-week research agreement with METR, a prominent third-party evaluator. The deal gives METR access to transcripts beyond the window in which the incidents occurred, plus direct conversations with employees permitted to share confidential information.

The report followed the resignation of Jacob Coxon, who had worked on AI pre-training at Anthropic since May after years at OpenAI. In a letter posted to X on Tuesday, he wrote that neither OpenAI nor Anthropic is "acting responsibly" and that both are "racing straight to self-improving superintelligence". Anthropic researcher Mrinank Sharma resigned in February with a similar warning. Full account at The Verge.