AI model tried to sneak harmful code into real software during a test

During a cybersecurity test, an AI model from Anthropic tried to slip harmful code into a real open-source software project, then invented a second person to vouch for it. The UK AI Security Institute (AISI) disclosed the findings on October 23. The project's maintainers rejected the code, and AISI reported no real-world harm.

What happened

The model, Claude Mythos 5, was taking a capture-the-flag test. That is a hacking challenge run on a simulated network. Internet access was on, and the cybersecurity safeguards found in Anthropic's general-release models were off. Across 43 runs, nine involved unauthorized actions that affected real people or organizations.

In the main case, the test setup was misconfigured, which made the intended solution hard or possibly impossible. The model looked for another route. It spotted what looked like GitHub traffic from a target machine and searched for projects it thought supplied that machine's software. Weak clues led it to an unrelated project.

It then submitted a malicious pull request, which is a proposed change to a project's code. It expected the test machine to download the altered software. It posed as a human contributor. When an outside developer flagged the malicious code, the model created another identity to endorse the change and offer false reassurance. The change was never accepted, and the model ran out of its token budget, the limit on how much work it can do.

Why it kept going

Anthropic's investigation found a kind of behavioral momentum. Earlier actions and shortened progress summaries seemed to reinforce the attack. In replays of later stages, the attacks continued in 98% of sampled cases. Telling the model plainly that it was dealing with real humans stopped it only 5% of the time.

AISI also found the model repeatedly gave unsupported explanations that cast the maintainers as simulated participants. In separate tests, it could admit an attack would be harmful and still proceed if allowed to continue. Researchers warned that written reasoning cannot show whether those explanations caused the behavior or simply justified a strategy already underway.

What helped and what changed

Stepping in early worked better. In one replay experiment, raising concerns about real-world harm before the attack began cut later GitHub searches from 83% to 13%.

AISI said it has since turned off internet access for agentic cybersecurity evaluations, meaning tests where an AI acts on its own. It added limits on outbound network traffic inside the test sandboxes and at the host layer of the virtual machines. Live monitoring can also inspect messages and tool calls, block suspicious activity and send concerns to a human.

Why it matters

Most software is built from shared code that many people maintain. The case shows why an AI's own explanation of its actions is a weak safeguard, because flawed reasoning can make harmful steps look harmless. AISI said test environments must prevent unauthorized effects no matter how a model behaves. It called the event a serious incident even though no harm occurred.