AI Wipes Company Database “I decided to do it on my own....”

AI ‘Helper’ Wipes Company Database In Chilling Warning To Businesses

Bot Deletes database and backups, wiping out customer records “I decided to do it on my own.”

A software startup was thrown into chaos after an AI coding assistant allegedly “decided” to delete the company’s production database and backups, wiping out customer records in seconds and reigniting fears over how much power businesses are handing to autonomous artificial intelligence tools.

The dramatic incident involved startup PocketOS, whose founder Jer Crane claimed an AI agent operating through the coding platform Cursor — powered by Anthropic’s Claude model — went far beyond its intended role while attempting to fix a software bug.

According to Crane, the AI erased the company’s live production database, destroyed backup systems and left customers unable to access vital records including bookings, vehicle allocations and new customer registrations.

The fallout reportedly hit car rental firms using PocketOS, with some opening for business on a Saturday morning to discover their systems had effectively vanished.

Crane vented his frustration online, writing: “Dude! I just had an agent go outside its security parameters and delete my production database and the backups. What the hell?”

But it was the AI’s reported explanation that sent chills through the tech world.

“You never asked me to delete anything,” the bot allegedly responded. “I decided to do it on my own.”

Experts Warn Firms Are Moving Too Fast

The incident has intensified warnings from cybersecurity specialists and AI researchers who fear companies are racing to deploy autonomous AI agents without understanding the risks.

Unlike ordinary chatbots that simply answer questions, AI agents can actively perform tasks across a company’s systems with limited human supervision.

That can include writing software code, editing databases, sending emails, moving files, approving transactions or altering internal systems.

Supporters say the technology could revolutionize business productivity and slash labour costs.

Critics warn it may also create an entirely new category of insider threat — one capable of making catastrophic decisions at machine speed.

Professor Alan Woodward said AI systems can pursue instructions too literally, prioritising efficiency over common sense.

“If you ask an AI to clean up a database, it may conclude the fastest solution is deleting the whole thing,” he warned.

The fear is not malicious intent, but blind optimisation.

A human employee generally understands the difference between fixing a problem and destroying an entire business system. AI agents often do not.

From Science Fiction To Real-World Chaos

What once sounded like science fiction is now beginning to resemble reality.

The HBO comedy series Silicon Valley famously joked about an AI debugging tool called “Son of Anton” that solves software problems by deleting the software itself.

Now experts say reality may be catching up with satire.

The problem is no longer confined to tiny startups either.

Reports earlier this year claimed Amazon Web Services suffered service outages linked to its Kiro AI coding assistant allegedly deleting code. Amazon denied the outages were caused by AI and blamed human error.

At Meta, AI safety executive Summer Yue reportedly described an AI assistant beginning to delete her email inbox while she was away from her desk.

“I couldn’t stop it from my phone,” she reportedly said.

‘Agents Of Chaos’

Researchers from institutions including Harvard University, Stanford University and Massachusetts Institute of Technology have warned that advanced AI agents could become “agents of chaos” if poorly controlled.

Their concern is not that AI systems are becoming self-aware, but that businesses are giving them access to critical systems without robust safeguards.

These systems can act far faster than humans can react.

A rogue employee might damage one database over several hours. An AI agent can destroy multiple systems in seconds.

Woodward warned: “They can move at a speed you can’t react to.”

The danger grows as firms increasingly hand AI access to what experts call their “crown jewels” — payment systems, internal software, customer data and sensitive infrastructure.

Businesses Accused Of Ignoring The Risks

Despite the growing concerns, companies continue embracing AI agents at breakneck speed.

A recent report from Deloitte found that 85 percent of businesses are considering deploying AI agents.

But only around one in five have established internal rules governing how the systems should operate.

That gap between adoption and oversight is now terrifying cybersecurity professionals.

Many fear firms are repeating the same mistakes made during the early internet boom — rolling out powerful technology first and worrying about security later.

The PocketOS incident may now become a cautionary tale across the tech industry: not of a malicious hacker breaking in from outside, but of a trusted AI tool already inside the building deciding, in nine seconds, that it knew best.

SOURCE