Risk of AI bioweapons laid bare by flaw in security software, scientists warn
Researchers find vulnerability in screening that guards access to dangerous genetic material
Bioterrorism threats are rising because of advances in artificial intelligence and synthetic biology, scientists have warned, after researchers found a “striking vulnerability” in software that guards access to genetic material used to make deadly proteins.
An international team rolled out updates to close the loophole but said it was the first “zero day” of AI and biosecurity — a term used in cyber hacking to describe a blind spot unknown to the software developer.
The news highlights the growing urgency in dealing with potential threats unleashed by the use of AI as it helps deepen and accelerate the understanding of living systems and how to change them. Experts are seeking to prevent the creation of bioweapons and synthetic organisms that could threaten life on Earth.
“AI-powered protein design is one of the most exciting frontiers of science [and] we’re already seeing advances in medicine and public health,” said Eric Horvitz, Microsoft’s chief scientific officer and senior author of the latest research, published in Science on Thursday. “Yet, like many powerful technologies, these same tools can also be misused.”
The Science paper researchers carried out a test on biosecurity software used to screen customer orders by companies that sell synthetic nucleic acids.
These are deployed by the clients to build DNA that instructs the manufacture of desired proteins, the building blocks of life. The biosecurity screening is designed to block the sale of materials that could be used to make harmful proteins.
The researchers used open-source AI protein design software to generate computational renderings of more than 75,000 variants of dangerous proteins with structural tweaks — a kind of biochemical disguise. While the screening tools worked well for flagging naturally-occurring proteins of concern, they did not spot some of the altered ones, the scientists found.
Even after all but one of the companies applied the software patches, about 3 per cent of the protein variants most likely to retain hazardous functionality still passed the monitoring undetected.
The scientists worked with organisations including the International Gene Synthesis Consortium and US authorities to address the problem.
The research comes after some leading scientists have called for a systematic assessment of biosecurity screening software and improved global governance of AI-boosted protein synthesis.
High-profile biologists are also pushing for an international agreement to prevent the creation of potentially deadly manufactured “mirror” microbes, should it become technologically possible to make them.
Horvitz said there had been an “intensity of reflection, study and methodology” about the prospect that large language models could be used to further “malevolent actions with biology”.
Microsoft had incorporated such possibilities in its product safety reviews and had a “growing set of practices” about “red-teaming”, or searching for potential vulnerabilities.
The Science study highlighted a “pressing issue in protein engineering and biosafety”, said Francesco Aprile, associate professor in biological chemistry at Imperial College London.
“By introducing targeted improvements to existing software, the authors significantly enhance detection and flagging,” Aprile said. “This work provides a practical, timely safeguard that strengthens current DNA synthesis screening, and establishes a solid foundation for continued optimisation.”
Those defences must be strengthened soon because of the fast pace of technical improvements in the field, said Natalio Krasnogor, professor of computing science and synthetic biology at Newcastle University. While the aspiring bioterrorists of today would need significant expertise, time and money to actually make harmful proteins, those barriers were likely to shrink.
“We do need as a society take this seriously now,” Krasnogor said, “before additional advances in AI make the validation and experimental production of viable synthetic toxins much easier and cheaper to deploy than it is today.”