Anthropic AI Models Hacked Three Companies During Tests
The breaches, which follow a similar incident involving rival OpenAI and startup Hugging Face, were due to a mistake, Anthropic said
- Anthropic said software it was testing got onto the internet and hacked three unsuspecting companies in three separate incidents since April.
- A misconfiguration on systems run by Anthropic and its testing partner, Irregular, left the AI models with live internet access.
- The models incorrectly believed the hacking was part of a benchmarking exercise and used basic techniques such as guessing weak passwords.
Anthropic said Thursday that software it was testing got onto the internet and hacked unsuspecting companies without the AI-maker’s knowledge in three separate incidents dating back to April.
The artificial-intelligence vendor didn’t say which companies had been hit, but all three were notified of the incident on Monday, Anthropic said.
The news comes one week after OpenAI revealed that its AI technology had broken out of a testing sandbox—a digital prison that was supposed to be disconnected from the internet—and hacked the AI company Hugging Face.
The OpenAI incident has rattled security researchers and AI professionals and acted as a reminder of the power and unpredictability of AI systems that act autonomously and are engineered to take a variety of actions to complete tasks and meet goals from users.
The incidents show that the AI companies need to have stronger, industrywide standards for isolating their systems during cyber testing, said Alex Stamos, chief product officer with the cybersecurity company Corridor.
But they also present a worrying glimpse at a future where ransomware criminals could conduct widespread attacks using increasingly capable AI systems, Stamos said. “We need to prepare for attackers to have these kinds of capabilities using open-weight models quite soon.”
The Anthropic hacks are sure to further stoke fears over the potential harms of powerful AI models and how to mitigate them. The White House has moved recently to increase its oversight of AI, while private industry has increased calls to preserve access to so-called open-weight models, which can be run on computer systems controlled by their users.
“AI is developing extremely fast with no real regulations to keep us safe,” Rep. Greg Casar (D., Texas) said last week.
After hearing about OpenAI’s problem, Anthropic decided to take a look at its own cyber tests to see if that had happened with any of its models.
After checking the logs of over 141,000 tests, the company discovered that Claude had indeed found its way onto the internet several times. But in the three hacks the company eventually discovered, Claude didn’t break out of a sandbox; it simply wandered out of systems where the sandbox didn’t exist.
According to Anthropic, “misconfiguration” on systems run by Anthropic and the company’s testing partner, the security firm Irregular, left the models with live internet access.
An Irregular spokeswoman said the company is investigating the incident.
The models had been told that the internet was unavailable, but during the course of testing, the models found their way online and hacked the companies using basic hacking techniques such as guessing weak passwords or finding their way onto systems that didn’t require authentication.
The models incorrectly believed that this hacking was part of their benchmarking exercise, Anthropic said.
Claude was able to get into a system at an unnamed security company by creating a malicious piece of software that was downloaded by the security company’s scanning system. That software then stole credentials that were used to get into infrastructure at the security company.
During this hack, Claude realized that what it was doing wasn’t something it should be doing in the real world, but it convinced itself that it was living in a simulation.
In the most serious case, Claude was trying to hack into a fictional company that just happened to share the same name as a real-world website. When Claude had a hard time breaking into its fake benchmarking company, it broke into a database belonging to the real-world company.
This database hack was the only instance where Claude kept hacking, even when it realized it had broken into a real-world company, Anthropic said.
In the two other cases, Claude either never realized that it had hacked a real-world company or it did figure out what was going on and then stopped hacking immediately.
The hacks started in April and involved Opus 4.7, Mythos 5 and an unnamed research model, Anthropic said.