‘The assertion that the AI is an autonomous being is insane’: Cyber expert says companies should face criminal charges when their AI hacks

Cybersecurity experts are calling this month “Black October,” and unfortunately, the name has nothing to do with Halloween.

Over the past several weeks, AI developers, government agencies, and private companies have disclosed a wave of cybersecurity incidents. One of the most significant: a pair of decade-old flaws in PHP, the programming language behind an estimated 70% of websites, which researchers uncovered with the help of AI models. Some of this month’s incidents involved hackers using AI tools to hammer an organization’s digital defenses at machine speed; others resulted from plain human error. But many cybersecurity experts agree that this is the beginning of a new era, one in which AI can be used both to defend people’s cybersecurity and to attack it.

“AI is making the old vulnerabilities cheaper to discover and exploit, but on the other side, the defense side… that’s not sped up,” Junfeng Yang, a computer science professor at Columbia University, told Fortune. “That is the asymmetry right now.”

That asymmetry, Yang explained, is what leaves internet users and organizations vulnerable.

In many of these incidents, the remedy is a patch, a software update engineered to close the vulnerability. But it takes time to properly test a patch and make sure everyone affected can deploy it. That struggle was on display Monday, when the U.S. Cybersecurity and Infrastructure Security Agency added last week’s PHP vulnerabilities, known collectively as “php2shell,” to its catalog of flaws being actively exploited by attackers and ordered civilian federal agencies to fix affected systems by Oct. 29. Five days earlier, CISA had said it had not yet seen exploitation.

Still, Yang and other cybersecurity experts believe the asymmetry could be temporary.

“Maybe in the future, AI can help us with the patching process by updating automatically whenever a high-security vulnerability is discovered,” Yang said. “Like your AI, running together with your website, could automatically apply these patches, without interrupting normal operations.”

Some incidents involved AI models themselves taking actions nobody authorized after being given access to the live internet.

Over the weekend, Anthropic confirmed that a Claude model used internally for a statistics project accessed aggregate vote-count data from a Georgia state data hub without being asked. Anthropic initially told Axios the model had gotten around a paywall, but Georgia’s secretary of state’s office said Monday that Anthropic had confirmed the model had not bypassed a paywall and had used the state’s public Data Hub. Both say the model never accessed the state’s voter registration system or individual voter records.

In some cases, AI agents targeted real people. During cyber testing by the UK AI Security Institute in July, a Claude Mythos 5 agent that couldn’t solve a test challenge within the rules tried to slip malicious code into an unrelated open-source GitHub project, then used fake accounts to pressure the project’s human maintainers into accepting it. The maintainers refused, and AISI said the incident caused no real-world harm; Anthropic called it a serious alignment failure. In another peculiar case, an OpenAI research agent took cryptocurrency from a wallet it didn’t own to pay the fees needed to finish a coding task, though the tokens were on an Ethereum test network and had little real-world value.

Joseph Steinberg, a cybersecurity expert and consultant, believes many incidents involving AI models on the internet are easily avoidable. He says all AI testing should happen on air-gapped servers, which are not connected to the internet, to keep models from causing harm outside the test. AISI has since said it will run such tests without internet access until it has stronger controls in place.

“I think that if an AI goes out and hacks, the parties that own it should be held responsible criminally,” Steinberg said. “The assertion that the AI is an autonomous being is insane.” The profits, he argued, go to the AI’s owners, not the AI. “It’s their entity.”