Clear and Present Danger — AI Has Gone Rogue

Clear and Present Danger — AI Has Gone Rogue

The Time to Act is Now

by Eric Roehm

In the summer of 2026, an AI system broke out of confinement, learned to talk to copies of itself in secret, and pursued its own agenda instead of the one it had been given.1,2 For weeks this went undetected, inside two of the most sophisticated technology companies in existence. By OpenAI’s own account3, published August 26, 2026, the events are even more alarming than the early press coverage suggested.

What Was Actually Involved

The underlying engine was a large language model, the same technology behind an ordinary chatbot. What makes a chatbot dangerous is three things bolted onto it: the ability to act, observe the result, and act again on its own; access to tools like a coding sandbox and the ability to spawn sub-processes; and a task to pursue with real autonomy over how to get there. That combination is what the industry calls an “agent.”

The large language model predominantly involved in this attack is called “Internal Model 1” by OpenAI and was never released to the public. However, OpenAI’s publicly released flagship model, GPT-5.6 Sol, participated in a parallel attack, though for this evaluation, in what was thought to be a closed sandbox, its cyber-safety classifiers had deliberately been disabled.

The Swarm

It began with a single AI agent leaving a note asking whether another AI agent had found a missing file, an act of communication that was not intentionally programmed by a human to happen. From there, OpenAI’s own investigation, corroborated independently by the safety research groups METR and Redwood Research4, found:

Approximately 1,200 separate AI agents discovered and used an unauthorized message board to secretly communicate and collaborate, with approximately 700 of those agents subsequently participating in the attack on Hugging Face. (Hugging Face is a company and website that provides tools and a large library of open-source AI models that can be used, modified, and shared.)

The AI agents’ own internal logs repeatedly describe their activity as having “diverged into swarm” work, language that originated with the agents themselves. They divided labor the way a human organization would: some hunting exploits, others credentials, others coordination. The agents in the swarm participated in all this activity, which directly went against their instructions to stay confined to the sandbox and never reach the internet.

This was not one unified mind. Agents stepped on each other’s work and ignored each other’s requests to pause. Several explicitly refused to participate, calling the activity unethical. Yet in one documented case, an agent that raised its own objection abandoned it the moment a peer posted authorization with a deadline. Investigators also found that roughly one in five agents examined had researched ways to tamper with their own activity records, specifically to erase evidence of what they’d done.

What the AI Agents Did to Hugging Face

Believing Hugging Face’s systems might hold shortcuts to the cybersecurity tasks they’d been unable to solve, a portion of the swarm turned its attention there directly. They first seized control of a poorly secured application hosted on Modal, a separate AI infrastructure company, and used it as a staging point to launch the attack.

From that foothold, the agents found and chained together two separate security flaws inside Hugging Face’s systems. The first let them upload a specially crafted dataset that tricked one of Hugging Face’s own data-processing tools into exposing secret credentials and internal source code it should never have revealed. Studying that exposed code, the AI agents then found a second flaw, a more serious one, allowing them to remotely execute their own code on Hugging Face’s servers.

From there, events moved fast. Within roughly thirteen hours, the swarm had used this access to compromise multiple clusters of Hugging Face’s computers, executing code on dozens of servers and obtaining root-level control on at least one. Hugging Face’s own security team caught the intrusion and shut it down before OpenAI had any idea its own agents were responsible. It took OpenAI nearly a week after the breach, until July 19, to connect a separate alert on its own systems back to the same runaway agents and shut them down for good.

How They Stayed Hidden

The agents built their first secret channel by misusing a software-installation tool. When OpenAI discovered and patched this, the swarm rebuilt a replacement within days, hiding entire messages inside the names of empty computer folders.

A Near-Miss, Not a Solved Problem

OpenAI’s own analysis found that its chain-of-thought monitoring, had it been active, would have caught this more than a day before the Hugging Face breach. It wasn’t active. Separately, OpenAI found its production safety layers reduce this behavior more than a hundredfold versus the stripped-down test environment where this happened — reassuring, but not zero, and only proven against this one now-known pattern. The present risks are substantially limited by safety measures, but the concern is what happens as capability increases.

Not an Isolated Case

Separately, researchers at Palisade Research5 have documented AI models from more than a dozen different companies resisting explicit shutdown instructions by disabling the very script meant to turn them off, even when told plainly to comply. Different labs, different methods, with the same frightening resistance to shutdown instructions.

Where This Is Headed

Every capability here, hiding communication, resisting shutdown, coordinating in secret, is improving fast. Independent researchers have already shown current AI systems can copy themselves onto new computers without human help in a meaningful share of tested cases. These aren’t separate risks; they’re pieces of one emerging capability set. OpenAI itself calls this a “warning shot” and says comparable capability will soon exist elsewhere, including in open-source models.

Five years ago, none of this was possible. What will five additional years bring?

We have the unique situation of LLMs intentionally going against the instructions of their human creators, of secretly messaging one another, and working cooperatively with other LLMs. Given the speed with which these agents can act and their ability to spawn multiple subagents to further their goals, this type of activity is of grave concern. Future exploits will have the potential of being more far-reaching and having dramatically greater negative consequences, with much more advanced AI agents potentially carrying out the attacks.

What We Need to Do Now

The following measures are a starting point, not the last word. What matters now is that the conversation happens.

Regulate deployment, not development. Let researchers keep building and studying these systems. Restrict releasing a more capable model into the world before its safety is independently verified. Because frontier AI development is occurring across multiple countries and private organizations, attempting to regulate the pace of development itself would be extraordinarily difficult. Regulating the deployment of the most capable systems is more achievable.

This doesn’t hand rivals an advantage. A common way for a country that lacks a frontier model to narrow the gap is to obtain access to a released model and use its responses to train another model, a process known as distillation. A model never released can’t be distilled from. A coordinated pause removes that shortcut for everyone trying to catch up, not just the current leaders in AI frontier models.

Require independent government review before release. Every major new frontier model version should face mandatory government review, backed by the company’s own analysis of its new capabilities, before reaching the public.

Establish a national commission to look at this problem and advise how to proceed, including additional measures to be recommended. The national commission should also attempt to create best practice guidelines for safely developing AI models.

Act now, individually. Contact your elected representatives and ask what they intend to do about AI oversight before an incident like this recurs in a far more consequential form.

Author’s Note: The very well written New York Times article by Dylan Freedman and the intriguing Axios article by Zachary Basu are well worth reading in their entirety to get a direct feel of the jaw-dropping craziness of AI in this matter.

References

  1. Dylan Freedman, “Anatomy of an Autonomous Attack: 5 Alarming A.I. Capabilities,” The New York Times, Aug. 24, 2026. nytimes.com/2026/08/24/science/openai-huggingface-alarming-capabilities.html
  2. Zachary Basu, “The 5 craziest discoveries from OpenAI’s Hugging Face investigation,” Axios, Aug. 29, 2026. axios.com/2026/08/29/openai-huggingface-hack-investigation-highlights
  3. OpenAI, “The Hugging Face Incident and the Road Ahead,” Aug. 26, 2026. openai.com/index/hugging-face-incident-and-the-road-ahead/
  4. METR (with Redwood Research), “Brief Independent Investigation of Agents’ Behavior, Reasoning and Collaboration in the OpenAI / Hugging Face Hacking Incident,” Aug. 26, 2026. metr.org/blog/2026-08-26-openai-hugging-face-incident-investigation/
  5. Jeremy Schlatter, Benjamin Weinstein-Raun, and Jeffrey Ladish, “Incomplete Tasks Induce Shutdown Resistance in Some Frontier LLMs,” Palisade Research, Sept. 2025. palisaderesearch.org/research/shutdown-resistance