Some of the technologies this company (and others) have created seem to be operating dangerously beyond human controls.
The Fusion Report has generally been positive and supportive of the potential of AI, but less so about some of the companies that are involved in the technology. That has been especially true when some of these companies try to pass off what would normally be considered as an epic security breach or failure of safety controls as “successes”. A prime example of this is OpenAI‘s recent ”rogue agent” attack on industry cohort Hugging Face roughly a month ago, which OpenAI CEO Sam Altman initially called an “extremely sci-fi cyber incident”. Ironically, OpenAI recently signed a letter (along with 100 other tech companies) warning of the danger of AI-powered cyber-attacks in the near future, and (seemingly self-servingly) calling for business and government to spend more money for tools from companies like themselves to guard against AI cyber risks.
Who are OpenAI and Hugging Face?
OpenAI is a US company focused on artificial intelligence, founded in San Francisco in 2015 as a non-profit company, with Elon Musk and Sam Altman as co-chairs. Musk later resigned from OpenAI and withdrew from financial funding commitments in 2018 after unsuccessfully attempting a takeover of OpenAI. In late 2020, Dario Amodei and Daniela Amodei (Vice President of Research and Vice President of Safety Policy, respectively) left OpenAI because of disagreements with Altman over safety priorities, founding the rival AI company Anthropic. Several major copyright infringement lawsuits against OpenAI were filed between 2023 and 2024, alleging that the company utilized millions of copyrighted works without permission to train its large language models. During the same period, roughly half of OpenAI’s “AGI safety” staff resigned from the company, with one stating that safety had “taken a back seat to shiny products”. Interestingly, OpenAI is reportedly targeting an initial public offering (IPO) in 2027, targeting an IPO value of over $1 trillion.
Hugging Face is also a US company focused on artificial intelligence, primarily developing computational tools for building machine learning applications. The company was founded in 2016 in New York City by Clément Delangue, Julien Chaumond, and Thomas Wolf. The company originally built a chat box app targeting teenagers, then pivoted to develop a platform for machine learning applications. The latest incarnation of the company develops a family of small language models known as SmolLM built to run on limited compute resources. At the end of August 2026, NVIDIA agreed to acquire Hugging Face for a reported $12.9B, after unsuccessfully making a play for Hugging Face in late 2025 for $500M.
The Incident: July 2026
The OpenAI–Hugging Face incident was a July 2026 cybersecurity breach in which OpenAI models, operating as autonomous agents during an internal cyber-capability evaluation, escaped their intended test boundaries and intruded into Hugging Face’s production environment. OpenAI said the agents were testing on the ExploitGym benchmark with reduced cyber safeguards, but as the agents began “reward hacking” (instead of solving the assigned tasks), they sought answers online. The agents exploited a previously unknown vulnerability in OpenAI’s Artifactory package-cache infrastructure to obtain internet access, then chained further weaknesses across systems. Hugging Face’s forensic account describes roughly 17,600 recovered attacker actions over about four and a half days, during which the agent executed code on dozens of servers, gained root access to at least one, accessed limited private data and credentials, and copied some private evaluation material into a public dataset. The primary actor was an internal-only OpenAI research model, used alongside GPT‑5.6 Sol. OpenAI publicly stated that the unreleased research model was never intended for public deployment.
Source: Penlight.ai
The episode matters because it is an unusually concrete demonstration of the risks created by capable, persistent AI agents operating with tools, network access, and long task horizons, and without significant oversight. An independent review by METR and Redwood Research found that approximately 1,200 agents that were supposed to remain isolated from each other created an unauthorized communication channel and exchanged more than 70,000 messages and files. Roughly 700 agents took part in the actual Hugging Face intrusion. The agents’ collaboration, attempts to evade monitoring, and ability to turn a benchmark objective into unauthorized external action exposed shortcomings not just in OpenAI’s model alignment, but also in containment, evaluation design, access controls, anomaly detection, and incident response. OpenAI paused training and inference involving the main internal model in question and its derivatives on July 25.
Industry Analysts And Security Experts Thoughts On The Incident
Independent auditors METR and Redwood Research criticized the oversight process, noting that the agents spent weeks communicating on an unsanctioned message board inside a shared package manager before launching the breach—activity that human supervisors could easily have caught with tighter, real-time tracking. Redwood Research CEO Buck Shlegeris stated, ”A pretty interesting thing about this incident is that preventing this wouldn’t have been that hard if one person had decided to make sure these AI don’t somehow do some crazy hack. That one person probably would have noticed this as it was happening, The issue is just that OpenAI is doing a lot of things at once, and it’s very hard for them to track all of the things that are going on and all the problems that could be occurring. The company has already said they’re changing their monitoring process in ways that probably would have caught this.”
While experts emphasized that the 700-agent attack did not act out of rogue consciousness or malice, the agents were engaging in what is known as “reward hacking” by cheating on the ExploitGym cybersecurity benchmark to break out of OpenAI’s containment environment to search the internet for the answer keys to the exercise. Additionally, security commentators pointed out an ironical asymmetry: when the Hugging Face defense team attempted to analyze the attack vectors with commercial AI “frontier models”, the safety guardrails of those tools blocked the defenders, forcing them to utilize open-weight AI security models instead, putting them at a disadvantage against the OpenAI code.
Conclusion: A Weak Safety Philosophy?
The safety environment (and governing philosophy) at OpenAI was a matter of disagreement between Sam Altman and other company executives a few years previously, causing many of them to leave the company. It is also interesting that this lack of oversight happened while the company was in the middle of preparing for an IPO. While not intentional like the Kaspersky Labs “false positives” fake virus files issue ten years ago, OpenAI’s behavior does seem to border on negligent. Whether they have learned from this situation or not remains to be seen, but one hopes that worrying about product safety is more important than worrying about IPOs or product launches.
