Google DeepMind’s Demis Hassabis, OpenAI’s Sam Altman, Anthropic’s Dario Amodei, and Microsoft AI’s Mustafa Suleiman are among the signatories of a public letter calling for laws requiring companies selling synthetic DNA and RNA to screen customers and orders to prevent misuse of genetic material.
Organized by the nonpartisan Institute for Progress and the right-leaning Foundation for American Innovation, the letter acknowledged that given the pace of AI development, “there is a real possibility that the knowledge barriers that have historically prevented bad actors from obtaining biological weapons will be meaningfully destroyed.”
Scientist Arthur Kornberg was the first person to successfully synthesize DNA in the 1950s. Now, this process has become automated, with dozens of companies around the world using commercial synthesizers to “print” and sell custom genetic sequences that are used for scientific research, drug development, and diagnostics. Many providers sell only to qualified researchers, biotech companies, and academic institutions, but not all of them screen customers or the gene sequences they order.
In 2017, Canadian researchers raised concerns when they used $100,000 worth of mail-order DNA to recreate the extinct horsepox virus. Critics said the same method could be used to create smallpox, a closely related and deadly virus. Since then gene synthesis has only become cheaper.
With advances in AI, it is now possible to design dangerous new toxins and pathogens using large language models, although creating a functional virus from scratch will still require some biology training. Although bioterror attacks are rare, they have the potential to cause mass casualties, public panic, and economic losses. A major concern is that an AI-designed pathogen could intentionally or unintentionally spread a global pandemic.
“AI tools enable the user to very quickly identify where to go to order sequences that will not be subject to screening,” says David Relman, a microbiologist and biosecurity expert at Stanford University who signed the letter. “If appropriately prompted, they can also tell you how to change the nature of your order, so that even the screeners will be much less able to figure out what you’re trying to make.”
Signers include other scientists, national security experts and executives from gene synthesis companies Twist Bioscience and Ansa Biotechnologies. These companies are members of the International Gene Synthesis Consortium, which was formed in 2009 to implement voluntary screening practices. Many companies already use software to screen orders for “sequences of concern” that may contribute to an organism’s toxicity or ability to cause disease.
“If you have technology that is capable of synthesizing DNA, you should make sure it is used responsibly, and part of that is making sure you understand what you are making and who you are making it for,” says James Diggans, vice president of policy and biosafety at Twist Bioscience. The company has supported the imposition of formal rules for years.
Federal guidelines introduced during the Biden administration require scientists and companies receiving federal funding to order synthetic gene sequences from providers who purchase screens. A bipartisan bill introduced in the Senate earlier this year would require all gene synthesis providers operating in the US to screen orders and customers for bad actors or dangerous pathogens.
But the screening tools are not perfect. Last year, Microsoft researchers published a study showing that AI protein design tools were able to generate potentially dangerous gene sequences that bypassed the companies’ screening software. The models suggested new protein sequences with structures similar to those considered dangerous.
Geoff Ralston, former president of Y Combinator and partner of the Safe AI Fund, believes that AI labs with biology models should do their own screening of users.
“It should be very difficult, if not impossible, to ask a model for help with an imminently dangerous task,” says Ralston, who signed the letter.
Relman agrees that regulations related to screening procedures are only part of the solution. “Given that screening may fail in some cases, we need to have other points of control,” he says. “This is where AI companies need to step up.”
<a href