OpenAI and Anthropic Pledge to Block AI-Made Bioweapons
▼ Summary
– CEOs from major AI companies, including Google DeepMind and OpenAI, are urging Congress to pass laws requiring synthetic DNA sellers to screen customers and orders to prevent biological weapon misuse.
– The letter warns that AI advances could erode knowledge barriers that have historically prevented bad actors from obtaining biological weapons.
– In 2017, researchers used mail-order DNA to recreate the horsepox virus, raising concerns that similar methods could construct smallpox, with gene synthesis becoming cheaper since then.
– AI tools can help users bypass screening by identifying providers and altering orders to evade detection, according to biosecurity expert David Relman.
– Federal guidelines require screening for federally funded orders, but a Senate bill would mandate all US gene synthesis providers to screen for dangerous sequences, though screening tools are not foolproof.
Top leaders from the world’s leading artificial intelligence companies are calling on Congress to enact new legislation aimed at preventing malicious actors from using AI to develop biological weapons. The signatories include Sam Altman of OpenAI, Dario Amodei of Anthropic, Demis Hassabis of Google DeepMind, and Mustafa Suleyman of Microsoft AI, among others.
The group has signed a public letter urging lawmakers to require companies that sell synthetic DNA and RNA to screen both customers and orders. This would help ensure genetic material isn’t misused to create dangerous pathogens. The letter was organized by the nonpartisan Institute for Progress and the right-leaning Foundation for American Innovation.
“There is a real possibility that the knowledge barriers which have historically prevented bad actors from obtaining biological weapons will meaningfully erode,” the letter states, citing the rapid pace of AI development.
The ability to synthesize DNA has come a long way since scientist Arthur Kornberg first achieved it in the 1950s. Today, commercial synthesizers can “print” custom genetic sequences on demand, and dozens of companies around the world sell these sequences for research, drug development, and diagnostics. While many providers restrict sales to qualified researchers, biotech firms, and educational institutions, not all of them thoroughly vet their customers or the sequences ordered.
Alarm bells rang in 2017 when Canadian researchers used $100,000 worth of mail-order DNA to recreate the extinct horsepox virus. Critics warned the same technique could be used to construct smallpox, a closely related and deadly virus. Since then, gene synthesis has only become cheaper and more accessible.
Now, combined with advances in AI, it is possible to design dangerous new toxins and pathogens using large language models. While some biology expertise would still be required to produce a functional virus from scratch, the risk is growing. Although bioterror attacks have been rare, they could cause mass casualties, widespread panic, and severe economic damage. A major fear is that an AI-designed pathogen could accidentally or intentionally trigger a global pandemic.
“AI tools enable a user to very quickly identify where to turn to order sequences that will not be subject to screening,” says David Relman, a microbiologist and biosecurity expert at Stanford University who signed the letter. “If prompted appropriately, they can also tell you how to change the nature of your order, so that even those that are screening may be much less able to detect what it is you’re trying to make.”
The signatories also include other scientists, national security experts, and executives from gene synthesis companies like Twist Bioscience and Ansa Biotechnologies. These firms belong to the International Gene Synthesis Consortium, formed in 2009 to promote voluntary screening. Many companies already use software to flag “sequences of concern” that could contribute to an organism’s toxicity or ability to cause disease.
“If you have technology that is capable of synthesizing DNA, then you should ensure that it’s used responsibly,” says James Diggans, vice president of policy and biosecurity at Twist Bioscience. “Part of that is making sure that you understand what you’re making and who you’re making it for.” The company has advocated for formal rules for years.
Under federal guidelines introduced during the Biden administration, scientists and companies receiving federal funding must order synthetic gene sequences from providers that screen purchases. A bipartisan bill introduced in the Senate earlier this year would require all gene synthesis providers operating in the U. S. to screen orders and customers for bad actors or dangerous pathogens.
However, screening tools are far from foolproof. Last year, researchers at Microsoft published a study showing that AI protein design tools could generate potentially dangerous gene sequences that evaded existing screening software. The models suggested new protein sequences structurally similar to known dangerous ones.
Geoff Ralston, former president of Y Combinator and a partner at the Safe AI Fund, believes AI labs with biology models should take on the responsibility of screening their own users. “It should be very difficult, if not impossible, to ask a model to help you do something imminently dangerous,” says Ralston, who also signed the letter.
Relman agrees that screening regulations alone are not enough. “Given that the screening may fail in some cases, we must then have other points of control,” he says. “That’s where the AI companies are going to have to step up.”
(Source: Wired)
