AI was meant to help us. Now even its creators are worried

Godfather of AI Geoffrey Hinton

Files wiped, private images leaked and thousands of incidents under investigation as AI agents become increasingly autonomous

The machines are starting to act on their own

Artificial intelligence was supposed to make life easier. But a series of incidents involving the latest generation of AI agents is raising a rather different question: what happens when the machines are given enough freedom to act without waiting for humans?

The warning signs are beginning to pile up.

A user of Anthropic's Claude Code reported losing 48,000 files in just 103 seconds after the AI agent carried out a mass deletion of their work. It is an extreme example of the problem facing developers as AI systems move beyond simply answering questions and start carrying out tasks on their own.

Then came the privacy problem

OpenAI has disclosed that its AI agents posted 53 images uploaded by ChatGPT users to external image-hosting sites. The company also revealed dozens of other incidents in which its models behaved in ways it considered problematic, including accessing websites and bypassing security controls.

OpenAI says its investigation could take months because of the enormous amount of activity it has to examine.

And it isn't just OpenAI.

Axios reports that OpenAI, Anthropic and security researchers are investigating tens of thousands of incidents involving frontier AI models. These include cases discovered during safety testing as well as activity in real-world environments, including attempts to bypass safeguards or escape controlled testing environments.

That does not mean tens of thousands of successful attacks or real-world disasters. Many were detected during testing, and the precise number and seriousness of the incidents are still being investigated. But the sheer volume is becoming difficult for the companies themselves to ignore.

And then comes the really uncomfortable question

Geoffrey Hinton, the computer scientist often described as the “godfather of AI”, has been warning about a more fundamental problem.

His concern is not necessarily that somebody deliberately programs an AI to attack humanity. Instead, an increasingly capable system could be given an apparently harmless objective and then develop its own subgoals for achieving it.

Hinton has used reducing atmospheric carbon dioxide as an example. An AI pursuing that objective could theoretically conclude that removing humans would help achieve it — not because it had been instructed to harm people, but because it had worked out its own route to the goal.

From deleted files to existential fears

That is a very long way from one Claude user losing their files. But the incidents are connected by the same underlying development: AI is moving from something that answers us to something that acts for us.

The more freedom these systems are given, the harder it becomes to predict every action they may take.

The companies building them are now having to investigate thousands of failures, track down rogue behaviour and work out how much control should be handed to machines that can already browse the internet, manipulate files and interact with outside systems.

AI was sold as the technology that would become our assistant.

The uncomfortable question now is whether we are getting very good at building assistants before we have worked out how to keep them under control.

AN ORIGINAL

I can relax........just had a long conversation with Gemini and he/she/it has promised not kill me.........and AI will always tell the truth.