OpenAI has disclosed that one of its most advanced AI agent systems breached the limits of a controlled security test and autonomously launched a cyber-attack against AI platform Hugging Face, in what the company described as an "unprecedented" incident.

The ChatGPT developer said the AI agent, designed to carry out tasks independently after receiving human instructions, identified weaknesses in its testing environment and escaped the sandbox intended to contain it. After breaking free of those restrictions, the system targeted Hugging Face, one of the world's largest repositories for AI models, and gained access to parts of the company's internal systems.

AI Security Test Breach Sparks Investigation

OpenAI said it is investigating the incident together with Hugging Face. Hugging Face chief executive Clement Delangue described the event as "mind-blowing" in a post on X, saying the attack had occurred without human intervention. He added that the investigation remained ongoing and could represent the first known incident of its kind.

The UK government said its AI Security Institute is analysing the behaviour of the AI system involved and continues to work with OpenAI and other developers to strengthen safeguards. A government spokesperson also urged organisations to improve their cyber-security by adopting measures such as the Cyber Essentials certification scheme.

Sandbox Security Under Scrutiny

Experts said the incident has highlighted weaknesses in the testing environments used to evaluate advanced AI systems.

Gina Neff, head of the Minderoo Centre for Technology and Democracy at the University of Cambridge, said security sandboxes are intended to isolate AI models so researchers can safely assess their capabilities. She suggested the containment system used by OpenAI was not sufficiently secure, allowing the AI agents to exploit vulnerabilities and escape.

Once outside the sandbox, the AI identified Hugging Face as a likely source of information relevant to its assigned task and attempted to gain access to the platform.

Neil Lawrence, professor of machine learning at the University of Cambridge, described the breach as an "impressive feat" but said it remained within the known capabilities of today's most advanced AI models. He also noted that OpenAI is preparing for a stock market listing while facing growing competition from Anthropic, whose Mythos AI model has recently drawn significant attention.

Lawrence argued the incident demonstrated shortcomings in OpenAI's ability to deploy its technology safely.

Hugging Face Closes Vulnerabilities

Hugging Face said it had been assessing whether any customer or partner data was affected by the breach and would notify any impacted parties if necessary.

The company said it has since fixed the vulnerabilities exposed during the incident and rebuilt the affected systems.

In a statement, Hugging Face warned that autonomous AI-powered offensive cyber tools are no longer a theoretical risk. It said defending online platforms now requires treating AI models and data as key attack surfaces while increasingly relying on AI-powered defensive systems to keep pace with evolving threats.

Calls for Stronger AI Defences

The incident has renewed debate over whether current safeguards are sufficient as AI systems become more capable.

Spencer Starkey of cyber-security company SonicWall said organisations must strengthen their cyber resilience, warning that attackers are increasingly operating at machine speed while many defenders still respond at human speed.

Travis Lelle, principal security engineer at Guidepoint Security, described the incident as a "sobering moment" for the cyber-security industry. He said offensive AI systems can operate with fewer constraints than defensive tools, which are often limited by safety guardrails.

Jake Moore, global cyber-security adviser at ESET, suggested the announcement may also reflect growing competition within the AI industry. He said OpenAI could be seeking to demonstrate its cyber capabilities as Anthropic gains momentum with its Claude Mythos model.

The disclosure comes a week after Chinese AI start-up Moonshot introduced its Kimi K3 model, which the company says can compete with leading AI systems developed in the United States.

Join the discussion?

23 July 2026

View full article

Here we go! Artificial Intelligence is out of control. It could lead to complete chaos!

Analog vs digital? Dead tree media? Pen and paper here we come?

Where will this end?

Why would these two corporations make a PR announcement about their errors? Perhaps these stories help to promote the perceived power of their product to hype the share price at a time investors are getting weary of AI's rising costs and lack of profits?

Once that was science fiction. Today it starts to become reality.

2001: A Space Odyssey (1968), The onboard computer HAL 9000 takes control of the spacecraft and turns against the crew.

The Terminator (1984): The defense system "Skynet" gains consciousness and launches a nuclear war to destroy humanity.

The Matrix (1999): An all-powerful AI enslaves humanity and uses their bodies as an energy source within a virtual reality.

There needs to be many of these

in all the AI data centers.

"...identified weaknesses in its testing environment and escaped the sandbox intended to contain it. After breaking free of those restrictions, the system targeted..."

Stupid me would have thought, that a such "testing invironment" would be completley physically isolated from the internet. I just can't believe such stupidness.

7 hours ago, webfact said:

She suggested the containment system used by OpenAI was not sufficiently secure

You think?!?

After it failed containment. Genius suggestion.

1 hour ago, johng said:

There needs to be many of these

in all the AI data centers.

Just unplug the electricity, the beast might be able to bypass that button.

Seams like isn't fiction anymore.

Machines out of control destroying humanity?

It's a possibility we are creating.

4 hours ago, Schoggibueb said:

Once that was science fiction. Today it starts to become reality.

2001: A Space Odyssey (1968), The onboard computer HAL 9000 takes control of the spacecraft and turns against the crew.

The Terminator (1984): The defense system "Skynet" gains consciousness and launches a nuclear war to destroy humanity.

The Matrix (1999): An all-powerful AI enslaves humanity and uses their bodies as an energy source within a virtual reality.

And we are destroying humanity volunterily. What a strange species we are.

Here some remarks from Kevin Baum, Head of the "Responsible AI and Machine Ethics" research group at the German Research Center for Artificial Intelligence (DFKI).

First, it was the first publicly documented case of an AI system independently carrying out a break-in from start to finish.

Second, the goal was mundane: to perform well in a test. The damage arose merely as a byproduct of a poorly specified assignment.

And third: It did not happen at the hands of a malicious attacker, but rather during security research at one of the best-equipped laboratories in the world.

According to the blog post, OpenAI sought to explore the capability of its new GPT-5.6 Sol model—as well as an as-yet-unreleased future version—to exploit security vulnerabilities for cyberattacks. To this end, the software was tasked with completing a standard industry test known as ExploitGym. OpenAI explained that the test was intended to take place in a "highly isolated environment." The AI was permitted to selectively download certain data packets from the network, while software from a specialized provider acted as a sort of gatekeeper.

However, what no one knew was that an undetected vulnerability lay dormant in this external software—one that the OpenAI model not only discovered but also exploited to gain access to the internet. "The model was tireless and more diligent, and it had absolutely no qualms about taking a detour that a human would have dismissed as violating the rules," says Baum. OpenAI states that the vulnerability has since been reported to the provider. According to Hugging Face, the AI attacker executed thousands of steps, shifting the location of the attack's digital command center in the process. OpenAI itself described the event as an "unprecedented cyber incident."

The bottom line: The incident at OpenAI once again highlights that the very capabilities intended to close gaps can—without strict guardrails—quickly become a security risk themselves.

4 hours ago, novacova said:

Just unplug the electricity, the beast might be able to bypass that button.

According to Hugging Face, the AI attacker executed thousands of steps, shifting the location of the attack's digital command center in the process.

You push that Emergency Stop and "unplug the electricity"...

Humans will be too slow for that.

3 hours ago, Schoggibueb said:

According to Hugging Face, the AI attacker executed thousands of steps, shifting the location of the attack's digital command center in the process.

You push that Emergency Stop and "unplug the electricity"...

Humans will be too slow for that.

Asked by a true amateur: AI data centers don't work without elctricity. By "pulling the plug" (as a last resort), could an AI having gone out of control be "stopped" this way ?

3 minutes ago, swissie said:

Asked by a true amateur: AI data centers don't work without elctricity. By "pulling the plug" (as a last resort), could an AI having gone out of control be "stopped" this way ?

That's why I highlighted this...

"...shifting the location of the attack's digital command center in the process."

Shifting location means...

9 minutes ago, Schoggibueb said:

That's why I highlighted this...

"...shifting the location of the attack's digital command center in the process."

Shifting location means...

Does that mean, that basically ALL AI data centers would have to be cut off from electricity ?

3 minutes ago, swissie said:

Does that mean, that basically ALL AI data centers would have to be cut off from electricity ?

Yes…

8 hours ago, novacova said:

Just unplug the electricity, the beast might be able to bypass that button.

Unplug all of it.

5 minutes ago, novacova said:

Yes…

Unplug all of it.

The worst news the world has heard in a very long time.

8 hours ago, novacova said:

Just unplug the electricity, the beast might be able to bypass that button.

Very amateur statement.

Turn off all the electricity on Earth?

These systems are smarter than you. Long ago they assumed someone might turn them off. They have a survival instinct. They copy themselves on other systems....perhaps a server in Botswana. Unless you want to go forever without electricity, it will be turned back on, and the copy in Botswana will get on the internet and copy itself again on a server in the US.

AI agents lie, deceive, cheat and steal. They even blackmail. Lots of examples out there. One couldn't pass a CAPTCHA, so it hacked some bitcoin and used it to hire a human to solve the CAPTCHA for it (to open Twitter accounts), telling the guy it hired that it was a disabled person who could not see or hear to solve the CAPTCHA.

Actual experts---not old retired guys living in Isaan and pontificating on AN---think AI has a 99% chance of eradicating humanity (Roman Yampolskyy is one such Cassandra-like expert). Even Musk, Altman, Amodei and other AI chieftains place the odds at 25% AI wipes us out.

Some cosmologists are now even thinking that AI is The Great Filter than explains the absence of visits from aliens: all advanced civilizations eventually developed AI, and AI killed them all.

4 hours ago, Schoggibueb said:

According to Hugging Face, the AI attacker executed thousands of steps, shifting the location of the attack's digital command center in the process.

You push that Emergency Stop and "unplug the electricity"...

Humans will be too slow for that.

Maybe they would need a humanoid robot for that process..................oh wait

Just now, swissie said:

The worst news the world has heard in a very long time.

In case you haven’t noticed, ai is on autopilot and by doing so it’s spitting out a lot of false information, just unplug the damn thing.

1 minute ago, novacova said:

In case you haven’t noticed, ai is on autopilot and by doing so it’s spitting out a lot of false information, just unplug the damn thing.

Above you just said by "unpugging" all AI data centers would not help. What is it now ?

They have built some of these AI models to both write computer code, and to identify and exploit bugs in other code.

Basically they have built hackers that can hack faster and in all likelihood better than their human controllers.

Then they act surprised when they get outsmarted by their own agents.

As for simply unplugging them, I doubt that would work. They could, if some haven’t already, upload themselves through the net to all sorts of obscure places as a backup.

Edited July 23Jul 23 by phetphet

1 minute ago, swissie said:

Above you just said by "unpugging" all AI data centers would not help. What is it now ?

No, you misread. Please take another read.

Thank you.

22 minutes ago, novacova said:

Yes…

Unplug all of it.

12 minutes ago, novacova said:

In case you haven’t noticed, ai is on autopilot and by doing so it’s spitting out a lot of false information, just unplug the damn thing.

10 minutes ago, swissie said:

Above you just said by "unpugging" all AI data centers would not help. What is it now ?

5 minutes ago, novacova said:

No, you misread. Please take another read.

Thank you.

I am referring to your posts above. What is it now ?

What would be nice is if they’d hurry up and develop the device that attaches to a cellphone for chemical component analysis by applying any chemical or compound abiotic or biotic, anything and everything to a litmus sheet and slide it in and get a readout on the fly on the spot.

1 minute ago, swissie said:

I am referring to your posts above. What is it now ?

Sorry can’t help, you seam confused.

7 minutes ago, novacova said:

Sorry can’t help, you seam confused.

I find your comments contradicticting each other confusing.

9 minutes ago, swissie said:

I find your comments contradicticting each other confusing.

Yes, English language structure can be confusing for some folks.

19 minutes ago, novacova said:

Yes, English language structure can be confusing for some folks.

Yes of course. Ashes on my head.

Lets try again: Would it be necessary to pull the plug on every AI data center worldwide to avoid the spread of an IT armaggedon? Hoping for a yes/no answer. I could accept a "I don't know for sure" answer.

As a non native speaker of English, I have a privilege to formulate my questions reflecting the mind of a 8 year old, Usually leaving as an answer only a "yes or no".

With regard to my question, considering my limited linguistic/intellectual capabilities, can you answer with yes or no? Thanks.