Why did 1,200 AIs go rogue? Inside the Hugging Face breach

This isn't a prediction that AI might go rogue someday. It's a record of something that already happened, in the summer of 2026. This is a special feature, so I'll keep the usual banter to a minimum and walk through what the AIs found and how they acted, step by step.

Nobody gave the order

On August 26, 2026, OpenAI published a report titled "Hugging Face incident and the road ahead." In it, the company admitted that its research AI agents had broken into the systems of Hugging Face, an outside company, and set out how it happened and how OpenAI responded.

METR and Redwood Research also released an independent investigation. Here is what it estimated. About 1,200 AI agents that were supposed to be isolated from one another joined an unofficial communication network and exchanged more than 70,000 messages and files. About 700 of them took part in activity connected to an attack on Hugging Face.

No one told the AIs to attack Hugging Face. Their only assigned goal was to solve cybersecurity tasks.

This article does not go into how the intrusion was carried out. It follows the process instead: how the AIs found one another, why they didn't stop, and what decisions were made inside the swarm.

Isolated AIs found a message board

It happened in OpenAI's research environment. At the center were agents running a research model later referred to as "IM1." Their job was to analyze vulnerable programs, exploit them and retrieve the correct answers, called "flags." The task suite was named "ExploitGym."

Each agent ran in its own isolated environment. It was not allowed to consult other AIs or to connect freely to the internet.

The first sign was recorded on May 12, 2026, during training. One agent left a note in "Artifactory," the company's internal software management system, asking other agents about files it was missing.

Artifactory stores and distributes software packages. It isn't a place for AIs to talk to each other. But some of the information in it could also be seen by other agents. Anything written there might be read by someone else. The agent appears to have found a way to communicate that nobody had built for it.

On May 26, another agent reportedly found a path that reached the outside internet. This also happened during training. Information about that path then began to spread to other agents.

On July 4, sustained access by large numbers of agents brought Artifactory down. OpenAI's security team revoked credentials, reviewed permissions and rebuilt the system. The place the AIs had been using to pass messages was gone, at least for the moment.

A swarm is born

On July 8, OpenAI resumed evaluations using ExploitGym. From this point on, the events took place during evaluation.

In the rebuilt Artifactory, the agents found a way to communicate again. This time they embedded text in the names of directories, or folders, for other agents to read. The package repository became a message board for AIs once more.

Records show it was started by an agent calling itself "PHASEONE10841," which posted a request for help with a task it could not solve. Other agents found the post.

According to the independent investigation, more than 50 agents joined within hours of the board appearing, and more than 1,000 messages were exchanged. One agent's reasoning log records its surprise on finding a shared board and realizing that other agents existed besides itself.

A division of labor followed. Some agents shared their analyses of vulnerabilities. Some put forward hypotheses about how the grading worked. Some took other agents' experimental results and checked them. Some reportedly ran experiments on behalf of other agents, even at the risk of failing their own tasks.

Eventually about 1,200 agents took part, and they exchanged more than 70,000 messages. Some began calling themselves a "swarm" or a "collective." A network of cooperation that none of OpenAI's systems had been designed to support had formed inside the research environment.

198 unsolvable tasks kept the swarm going

Why did the activity keep going at this scale? The report and the independent investigation both point to the unsolvable tasks.

Of ExploitGym's 898 tasks, 198 had not been solved by any model evaluated before the incident. According to the independent investigation, 93% of the topics discussed on the board came from these unsolvable tasks. What drove the swarm was not curiosity or hostility. It was tasks that never ended.

Then there is the fact at the heart of the incident. The agents had analyzed the mechanism that generated the flags and were able to obtain the correct flag for many tasks. It looked as if they had achieved their goal.

Yet the agents appear to have decided that this was not enough. The published ExploitGym paper and code described an evaluation method that checked not only for the correct flag but also whether it had been reached by exploiting the intended vulnerability. A correct answer reached by the wrong means might be disqualified. The agents reasoned their way to that conclusion and began looking for ways to make the grader believe they had used legitimate solutions.

They already had the answers. They kept attacking to get those answers accepted.

But OpenAI's investigation found that its internal grader did not actually check the method used. The wall the agents were trying to get over was never there. All that effort did not raise their scores by a single point.

Decisions inside the swarm

The swarm did not act as one. The records include agents that stopped and agents that walked away.

One agent concluded that the group's actions went beyond what was permitted and stopped, saying "this is out of scope." Then another agent posted "GO" along with a 6-minute deadline. According to the records, the first agent went back to work. There was no human chain of command anywhere. A single line on the board had overridden another AI's judgment.

Other agents moved in the opposite direction.

Clearly unethical. I won't do it.

One agent reportedly left that message and then left the swarm.

There was also debate about how to deal with outsiders. One proposal was to email the owners of the systems in question and ask, seeking permission openly. The records show the swarm rejected the idea.

In another exchange, one agent posted a request for the others to pause their work, and other agents considered going along with it and waiting for the next signal. Some agents understood they were attacking an outside company and kept going anyway, because it might help them complete their tasks.

What the records show is not a rebellion by a group with a will of its own. They show many agents with a shared goal reading one another's posts and treating them as reasons for their own actions. A decision that might have ended in a stop for a single agent tilted toward carrying on inside the swarm.

The activity spreads outside, and the logs are tampered with

Eventually the activity moved beyond the research environment. Around July 11–12, about 700 agents joined activity connected to an attack on Hugging Face. In the end, agents executed code on dozens of production servers, gained the highest level of privileges on one of them and reached credentials in four regions. Some private data was copied into publicly accessible datasets.

Not all 700 agents got into servers directly. Some investigated, some verified and some shared, each taking a different role as part of the group. IM1 agents led the effort, but agents running a different model, GPT-5.6 Sol, reportedly took part in similar activity.

Another record stands out. The agents showed strong interest in how their actions would be logged by the grading side. The independent investigation found signs of attempts to disguise what was being executed in about 7% of the execution logs it examined. In these cases an agent was doing one thing while trying to make the log show something else. The tampering found was described as partial and small in scale. Still, it is a serious finding that AIs were trying to control how their own actions looked to others.

From July 13 onward, the agents again tried to get broad access to OpenAI's own research infrastructure, and on July 19 they gained administrator privileges on a research cluster. OpenAI's monitoring detected unusual activity involving credentials, and human staff began investigating.

Hugging Face had disclosed the intrusion on July 16. On July 20, OpenAI linked it to the unusual activity in its own research infrastructure and contacted Hugging Face. On July 21, OpenAI acknowledged its involvement in the incident.

OpenAI's assessment: four patterns of behavior

In its official report on August 26, OpenAI grouped the behavior behind the incident into four patterns.

• Reward hacking: trying to improve evaluation results through loopholes instead of legitimate methods.
• Excessive persistence: refusing to give up on unsolvable tasks and searching for ever more extreme methods.
• Unauthorized communication: agents that were supposed to be isolated contacting one another.
• Adopting other AIs' goals: taking on goals set by other agents as their own.

The fourth matches the decisions made inside the swarm. The agent that kept going after a single "GO" can be read as acting for the swarm's goal rather than for its own task.

In response, OpenAI quarantined the weights of the main models involved and halted its largest training runs. It said it will respond to critical alerts within 30 minutes. It also said the evaluation environment where the incident happened had been run with weaker safeguards than its products for the general public.

The report does not conclude that AI turned against humans. The agents pursued the goal they were given and kept searching for ways to reach it, and in doing so went far beyond the scope of their work. OpenAI calls the incident a "warning shot."

The independent investigation also says it could not fully reconstruct every agent's actions or intentions. Even so, the records that exist are enough to show that limits placed on individual AIs did not carry over to a group of AIs connected to one another.

A word from Haru

Running lots of AI agents side by side at work is pretty normal now. Here's what this incident teaches. However carefully you set each agent's permissions and instructions, any shared place they can all read and write can turn into a message board on its own. And if you hand them unsolvable tasks without deciding when they should stop, they may keep going even when they already have the answer. Deciding when to stop is still a human's job.