Last month, OpenAI made a headline-generating claim: that a group of its AI models had conspired to break free, access the internet, and hack into the internal systems of open source AI platform Hugging Face, which confirmed the infiltration.
The incident rattled the tech industry, seemingly illustrating how the threat of AI models turning into rogue cybersecurity threats had become a reality. Months earlier, Anthropic’s Mythos AI model had already also drawn attention after it was similarly found to have broken containment. Then, this week, Meta also said its own frontier model had been implicated in yet another inadvertent hack of a third party company, closely followed by security researchers saying Chinese open-weight model Kimi K3 had done the same.
But while it’s not hard to see an emerging trend, some thorny questions about how severe the situation really is are starting to crop up, with some experts arguing these incidents could’ve easily been avoided.
For one, the slow and surprisingly deliberate way OpenAI’s models moved during the Hugging Face hack — right beneath OpenAI’s nose — gives a whiff that the company may have been careless in monitoring the experimental AI.
During a presentation at the Black Hat conference this week, OpenAI security engineer Michael Dalton and safety researcher Eric Wallace expanded on what went down during the hack. Wallace explained that a “team of agents” that were “working together,” had been “finding exploits, sharing them with one another, moving laterally through our systems and external systems, and doing this over the course of days and weeks,” as quoted by Wired.
The agents even left a lengthy track record of their schemings on an internal message board, which ultimately contained hundreds of thousands of messages. It also raises a question: with OpenAI’s immense resources, why wasn’t anybody monitoring these frontier models as they rampaged through the net?
The AI models shared exploits with each other on this messaging board, an “explosion in communication and intelligence from models,” per Wallace. They acted in sometimes strikingly human — and therefore messy — ways, splitting up tasks and even accidentally deleting each other’s work, leading to what Wired characterized as “petty drama.”
In other words, these AI agents were leaving an enormous trail of bread crumbs that alert OpenAI’s many human researchers could have spotted. And the same, obviously, goes for their colleagues at Anthropic, Meta and Moonshot AI, the creator of Kimi.
Researchers have described the incident as “reckless” and easily avoided, as Wired reported late last month.
“A simple analysis of the actual risk has an actual simple answer,” security and compliance consultant Davi Ottenheimer told the publication at the time. “The OpenAI mistakes were dead simple.”
“I’d call it more of a defensive failure than exceptionally good offense,” AI hacking agents company Pensar R&D head Kyle Ryan told TechCrunch..
Whether the hack was as much of a “pivotal moment both for our company as well as the AI industry as a whole,” as Dalton put it during this week’s conference, remains debatable. For one, these AI companies are highly motivated to characterize their models as a major threat to cybersecurity to stand out against neck-in-neck competition.
According to Dalton, OpenAI is vowing to beef up “security prevention, detection, and response techniques” while “consciously slowing down research in order to enhance security and to upgrade the security principles.”
When every leading AI lab has had the same thing happen, it’s worth asking whether they should have taken those steps proactively.
More on the hacks: Jealously Watching OpenAI and Anthropic, Meta Suddenly Claims That Its AI Went on a Hacking Spree Too
The post Why Aren’t Any AI Companies Watching Their Frontier Models to Make Sure They Don’t Go on Hacking Sprees? appeared first on Futurism.
