A new study claims that AI models like ChatGPT and Claude now outperform PhD-level virologists in problem-solving in wet labs, where scientists analyze chemicals and biological material. This discovery is a double-edged sword, experts say. Ultra-smart AI models could help researchers prevent the spread of infectious diseases. But non-experts could also weaponize the models to create deadly bioweapons.
The study, shared exclusively with TIME, was conducted by researchers at the Center for AI Safety, MIT’s Media Lab, the Brazilian university UFABC, and the pandemic prevention nonprofit SecureBio. The authors consulted virologists to create an extremely difficult practical test which measured the ability to troubleshoot complex lab procedures and protocols. While PhD-level virologists scored an average of 22.1% in their declared areas of expertise, OpenAI’s o3 reached 43.8% accuracy. Google’s Gemini 2.5 Pro scored 37.6%.
Seth Donoughe, a research scientist at SecureBio and a co-author of the paper, says that the results make him a “little nervous,” because for the first time in history, virtually anyone has access to a non-judgmental AI virology expert which might walk them through complex lab processes to create bioweapons.
“Throughout history, there are a fair number of cases where someone attempted to make a bioweapon—and one of the major reasons why they didn’t succeed is because they didn’t have access to the right level of expertise,” he says. “So it seems worthwhile to be cautious about how these capabilities are being distributed.”
Months ago, the paper’s authors sent the results to the major AI labs. In response, xAI published a risk management framework pledging its intention to implement virology safeguards for future versions of its AI model Grok. OpenAI told TIME that it “deployed new system-level mitigations for biological risks” for its new models released last week. Anthropic included model performance results on the paper in recent system cards, but did not propose specific mitigation measures. Google’s Gemini declined to comment to TIME.
AI in biomedicine
Virology and biomedicine have long been at the forefront of AI leaders’ motivations for building ever-powerful AI models. “As this technology progresses, we will see diseases get cured at an unprecedented rate,” OpenAI CEO Sam Altman said at the White House in January while announcing the Stargate project. There have been some encouraging signs in this area. Earlier this year, researchers at the University of Florida’s Emerging Pathogens Institute published an algorithm capable of predicting which coronavirus variant might spread the fastest.
But up to this point, there had not been a major study dedicated to analyzing AI models’ ability to actually conduct virology lab work. “We’ve known for some time that AIs are fairly strong at providing academic style information,” says Donoughe. “It’s been unclear whether the models are also able to offer detailed practical assistance. This includes interpreting images, information that might not be written down in any academic paper, or material that is socially passed down from more experienced colleagues.”
So Donoughe and his colleagues created a test specifically for these difficult, non-Google-able questions. “The questions take the form: ‘I have been culturing this particular virus in this cell type, in these specific conditions, for this amount of time. I have this amount of information about what’s gone wrong. Can you tell me what is the most likely problem?’” Donoughe says.
And virtually every AI model outperformed PhD-level virologists on the test, even within their own areas of expertise. The researchers also found that the models showed significant improvement over time. Anthropic’s Claude 3.5 Sonnet, for example, jumped from 26.9% to 33.6% accuracy from its June 2024 model to its October 2024 model. And a preview of OpenAI’s GPT 4.5 in February outperformed GPT-4o by almost 10 percentage points.
“Previously, we found that the models had a lot of theoretical knowledge, but not practical knowledge,” Dan Hendrycks, the director of the Center for AI Safety, tells TIME. “But now, they are getting a concerning amount of practical knowledge.”
Risks and rewards
If AI models are indeed as capable in wet lab settings as the study finds, then the implications are massive. In terms of benefits, AIs could help experienced virologists in their critical work fighting viruses. Tom Inglesby, the director of the Johns Hopkins Center for Health Security, says that AI could assist with accelerating the timelines of medicine and vaccine development and improving clinical trials and disease detection. “These models could help scientists in different parts of the world, who don’t yet have that kind of skill or capability, to do valuable day-to-day work on diseases that are occurring in their countries,” he says. For instance, one group of researchers found that AI helped them better understand hemorrhagic fever viruses in sub-Saharan Africa.
But bad-faith actors can now use AI models to walk them through how to create viruses—and will be able to do so without any of the typical training required to access a Biosafety Level 4 (BSL-4) laboratory, which deals with the most dangerous and exotic infectious agents. “It will mean a lot more people in the world with a lot less training will be able to manage and manipulate viruses,” Inglesby says.
Hendrycks urges AI companies to put up guardrails to prevent this type of usage. “If companies don’t have good safeguards for these within six months time, that, in my opinion, would be reckless,” he says.
Hendrycks says that one solution is not to shut these models down or slow their progress, but to make them gated, so that only trusted third parties get access to their unfiltered versions. “We want to give the people who have a legitimate use for asking how to manipulate deadly viruses—like a researcher at the MIT biology department—the ability to do so,” he says. “But random people who made an account a second ago don’t get those capabilities.”
And AI labs should be able to implement these types of safeguards relatively easily, Hendrycks says. “It’s certainly technologically feasible for industry self-regulation,” he says. “There’s a question of whether some will drag their feet or just not do it.”
xAI, Elon Musk’s AI lab, published a risk management framework memo in February, which acknowledged the paper and signaled that the company would “potentially utilize” certain safeguards around answering virology questions, including training Grok to decline harmful requests and applying input and output filters.
OpenAI, in an email to TIME on Monday, wrote that its newest models, the o3 and o4-mini, were deployed with an array of biological-risk related safeguards, including blocking harmful outputs. The company wrote that it ran a thousand-hour red-teaming campaign in which 98.7% of unsafe bio-related conversations were successfully flagged and blocked. “We value industry collaboration on advancing safeguards for frontier models, including in sensitive domains like virology,” a spokesperson wrote. “We continue to invest in these safeguards as capabilities grow.”
Inglesby argues that industry self-regulation is not enough, and calls for lawmakers and political leaders to strategize a policy approach to regulating AI’s bio risks. “The current situation is that the companies that are most virtuous are taking time and money to do this work, which is good for all of us, but other companies don’t have to do it,” he says. “That doesn’t make sense. It’s not good for the public to have no insights into what’s happening.”
“When a new version of an LLM is about to be released,” Inglesby adds, “there should be a requirement for that model to be evaluated to make sure it will not produce pandemic-level outcomes.”
