A disease doesn’t have to be real to cause worldwide damage
“While I am deeply concerned about the long-term existential threat of AI and synthetic biology to create new or modified pathogens, my extensive experience detecting and controlling outbreaks around the world makes me fear a more immediate threat: a rogue actor using existing AI tools to simulate a bioterrorism attack that would destabilize a region or the world. [..]
Freely available AI tools now permit people to create “deepfakes” that are almost impossible for a person to differentiate from reality without special tools. It’s not simply a question of whether you believe a conspiratorial idea. Do you also dismiss what your eyes and ears are telling you as well?
Let’s consider a scenario that keeps me up at night: two nuclear powers accusing each other of bioterrorism.
It’s July 2025. In their large group chats, WhatsApp users in India begin forwarding videos showing patients lying on medical cots, writhing in pain, while covered in a disfiguring rash that looks like smallpox. The videos say that these are images of real patients from a remote clinic on the Indian side of the Line of Actual Control with China, an area in the Himalayas in which there is no internationally recognized border and there are frequent armed conflicts.
The videos then start circulating globally on X, Instagram, YouTube, Facebook, and TikTok, accompanied by ‘“new” images of hastily assembled isolation wards outside a rural hospital in India. Multiple sources begin sharing similar content simultaneously, including members of parliament and influencers, creating an illusion of independent verification.
Voice recordings emerge of distressed medical staff, speaking in local dialects with hospital sounds behind them, saying they are overwhelmed by people coming to the hospital with fever and rash and that they fear for their own lives because they do not have protective equipment and are not vaccinated against smallpox. Soon after, video clips are posted of local government officials gathering for an emergency meeting to discuss this outbreak of smallpox, a disease previously eradicated. These officials discuss the need to “lock down” the community and have security services block any movement near the border with China.
In reality, the audio, pictures, and video have all been fabricated by an extremist group using widely available AI tools and exploiting social media algorithms. But no one is listening to the few experts who suspect these are deepfakes. The Indian military mobilizes more forces to the Line of Actual Control, interpreting the outbreak as a potential biological threat to national security. Local Indian administrators demand immediate access to Chinese medical facilities for inspection.
China refuses, saying it violates their sovereignty. They deploy military forces under the guise of “medical quarantine enforcement,” militarizing their response to a fictional threat. State media in both countries begin suggesting the outbreak was engineered by the other side: Chinese outlets begin calling it “the India virus” and say it likely started in Indian labs that cooperate with the U.S. on medical research, while Indian outlets point to Chinese facilities and say this is “just like what happened with Covid.”
Military commanders from both sides call for “preventive measures” to prevent another bioterrorism attack. Media and elected officials begin saying that a smallpox bioterrorism attack merits a nuclear response.
What are health officials both nationally and globally doing during this time? When learning how to respond to an outbreak, every field epidemiologist learns: first verify there actually is an outbreak. In this situation, that means deploying public health staff to the impacted area, examining patients, reviewing medical records, and obtaining specimens that can be tested in a high-quality laboratory.
Verifying the existence and cause of an outbreak is challenging even under routine circumstances in low- and middle-income countries. Consider the extensive media and global attention to a “Disease X” outbreak in the Democratic Republic of Congo in December 2024. Health officials and experts around the world sounded the alarm that there was another potentially new disease emerging in the DRC. And yet it took more than 2 weeks from the first media reports of Disease X until it was confirmed to not be “X.” Laboratory specimens from patients and a careful review of medical records showed that women and children were primarily dying from malaria and malnutrition — data that, in an ideal world, would have come from doctors doing routine lab testing. [..]
In the scenario I worry about, health officials would likely rely on their visual inspection of “patients” and preliminary lab results to conclude there is no smallpox outbreak, but security officials would likely require a much stricter standard of evidence, given their fear that being wrong could be seen as weak or making their country susceptible to further attack. Which country will be the first to acknowledge they were duped? Will citizens, already angry at the other side or at their own government, even believe a declaration that this was fake? Will military officials eager to resolve the border dispute use this as a pretext for military action?
While WHO and many health agencies have increasingly held meetings and issued guidance to reduce the risk of and manage accidental and deliberate biological attacks, I fear there is insufficient attention being paid to the more immediate threat of AI-fabricated outbreaks. Even if a scenario such as this occurred in a less tense region, it still has the potential to overwhelm health services, erode trust in institutions, and provoke unrest.
There is an urgent need for health and security agencies to raise awareness about this risk. Health and security agencies need to run exercises — as they often do already for suspected bioterrorism attacks — about how they will, in fact, verify whether audio and video reports are real and what standard of evidence they will use to make this determination. These exercises, however, cannot occur in isolation. They must be done in collaboration with leading technology and media companies as well to develop detailed protocols for how to screen and verify deepfakes related to infectious disease threats, and how those protocols can continuously be updated to keep up with the latest developments in AI. They must be discussed widely in the media to make the public aware of these scenarios. And clear protocols must be developed in government for security and health officials to feel comfortable admitting their errors if they initially declare an outbreak in response to a deepfake. [..]
While we are witnessing two scientific revolutions occurring at the same time — AI and synthetic biology — the threat of these technologies emphasizes, yet again, the importance of enhancing systems that already exist: training and hiring more epidemiologists to do rapid, high-quality investigations of suspected events, improving the clinical lab capabilities in low- and middle-income health care facilities to do testing for common infectious diseases, and funding public health systems and facilities to collect, transport, and test specimens for natural, accidental, deliberate, and fake threats.
The existential threat from AI is real. At this moment, however, it is the “fake” threats that I fear are most imminent.”
Full editorial, JK Verma, STAT, 2025.5.27