SAN FRANCISCO – Researchers at the University of Toronto say they have found a way to use artificial intelligence to create a dangerous computer “worm” capable of targeting any known flaw in the world’s computers and quickly spreading mayhem throughout the internet.
The computer scientists said in a paper published at night on June 2 that this programme could be built and that a prototype they had created spread across a test network with no human intervention.
The researchers kept their test network isolated from the public internet. They also redacted some details from the paper describing how they built the worm so that hackers would not be able to use the paper as a blueprint for attacks.
But their work is likely to raise fears that AI is leading to a new era of computer hacking that will be difficult to defend against. It also adds to growing evidence that advances in AI are creating risks to computer networks that would have been hard to imagine just a few years ago.
The AI company Anthropic said in April that its latest technology, Claude Mythos, was too powerful to share with the public because hackers could use it to exploit security holes in computer networks faster than they ever could before.
Anthropic limited the release of the technology to about 40 organisations that maintain critical computer infrastructure so they could use the system to patch security vulnerabilities before hackers took advantage of them.
A week later, OpenAI, Anthropic’s chief rival, said it was limiting the release of similar technology. OpenAI shared its new system with hundreds of organisations before expanding the release to thousands of partners in the weeks that followed.
(The New York Times sued OpenAI and Microsoft in 2023, claiming copyright infringement of news content related to AI systems. The two companies have denied those claims.)
The paper from the University of Toronto adds a new twist to AI fears. Because the AI technology that powered the worm was “open source” or “open weight” – meaning it has been freely shared on the internet – no one can restrict how it is used. The proverbial genie is out of the bottle.
“You have to have a perfectly secure system to defend against this – and we know that is not currently feasible,” said Nicolas Papernot, a professor of computer engineering at the University of Toronto who led the team that built and tested the prototype.
Papernot and his team, which published the paper on his lab’s website, were able to create what is essentially an AI-powered version of the computer worms that hackers started releasing onto the internet two decades ago. Unlike other kinds of computer viruses, worms spread from machine to machine on their own, without help from humans.
With names like SQL Slammer, Conficker and Stuxnet, each of these self-replicating software programmes exploited a specific vulnerability in computers, taking control of millions of machines, stealing their data, deleting their files and generally wreaking havoc.
After a decade of attacks, many computer users learnt to quickly patch their most glaring vulnerabilities. But the threat never went away.
In 2017, another worm, WannaCry, targeted another major flaw in the world’s machines and infected more than 300,000 machines in 150 countries, taking their data hostage and demanding bitcoin ransom payments.
The prototype built by the Toronto researchers takes this kind of self-replicating worm a step further. It can rapidly spread across a network by tailoring a new attack for each machine it encounters. As Papernot described it, the worm could “reason” through new attack strategies.
“This makes it significantly more difficult to stop the spread of malware,” he said. “There is no longer a single software fix you can apply to the devices to protect them from the worm.”
The worm could run on computers that use either the Windows or the Linux operating system. And although the worm, because of its complexity, cannot operate without finding a more powerful machine, it could attack less powerful machines on the same network, including laptops, printers and cameras.
Security experts are not surprised that AI can tailor attacks. Over the past year, companies in the United States, China and other parts of the world have built AI systems that are particularly good at writing computer code. If an AI system can write code, it can potentially exploit vulnerabilities in software applications.
But leading systems from companies like Anthropic and OpenAI cannot be packaged into worms because they are not open source and, in all likelihood, are too large to run on many computers. Many experts assumed that open source AI technologies were not powerful enough to drive self-replicating computer worms.
Some outside experts said the threat may be limited because AI systems are prone to mistakes.
“There is usually a meaningful gap between what you can create in lab conditions and what you can pull off in the world to create significant damage,” said Dan Lahav, chief executive of Irregular, a security company that specialises in threats from AI.
“AI systems tend to be unpredictable and clumsy,” he added. “They do weird stuff, and that can trigger security defences.”
But Lahav also warned that AI would continue to improve. That means companies have to patch as many software vulnerabilities as possible, and they can use AI to help do it.
For that reason, researchers said, Anthropic should share Mythos with a wider group so it can be used to fight AI threats. On June 2, Anthropic said it would share its technology with 150 additional organisations.
“Ultimately, broader distribution – so that people can use the technology to fix vulnerabilities – is the way to go,” said David Lie, a computer science professor at the University of Toronto who reviewed the paper but was not part of the team that built the worm. NYTIMES
