AI redesigned a toxin. Nobody made it.

Order synthetic DNA from a commercial provider and your sequence gets screened first. The provider runs it against databases of known sequences of concern, looking for close matches to things nobody should be mailing out. That screening is one of the few genuine choke points in biology. It is not a law of nature, it is software, and software has a threat model.

A team led by Bruce Wittmann and Eric Horvitz at Microsoft, working with the DNA synthesis industry, asked what happens to that software when the sequence on the order form was written by a generative protein model rather than copied from a pathogen. The answer, published in Science on 2 October 2025, is that the screening did not reliably catch it.

That finding got covered widely, and the coverage compressed it into something like "AI can now design bioweapons that evade detection." I want to walk through what was and was not established, because the gap between those two things is the whole subject of this beat, and because a follow-up study has since done the experiment the first one could not.

What was actually run

The team used open-source AI protein design software to generate variants of hazardous proteins, then submitted those sequences to the screening tools used by nucleic acid synthesis providers to see what got flagged. The AAAS release accompanying the paper describes more than 75,000 variants of hazardous proteins, evaluated against four biosecurity screening software developers.

Many got through. In response, the team developed patches for the screening tools, and three of the four screening providers deployed them. After patching, by the release's account, about 3% of the variants most likely to retain functionality still escaped detection.

This is a responsible-disclosure story and it should be read as one. Find a gap, tell the vendors privately, build the fix, ship the fix, then publish. That is how it is supposed to work, and the two-year timeline before publication reflects that. Nothing in this report is a criticism of how the work was handled.

The five words carrying the load

Here is the paper's own abstract, describing the outcome of the patching:

we developed and deployed patches, greatly improving detection rates of synthetic homologs more likely to retain wild type-like function.

Read "more likely to retain wild type-like function" carefully. That is a prediction about the variants, not a measurement of them. The sequences were ranked by computational estimates of whether they would still work as proteins. Nobody synthesized them, expressed them, purified them, or tested whether they did anything at all.

The paper is honest about this. The hedge is right there in the abstract, and Microsoft's own writeup describes the finding as coming from "computer-based studies." The imprecision entered downstream, when a screening-software vulnerability became a story about AI designing functional toxins.

The distinction matters because a protein sequence is not a protein. Rewriting a sequence so it looks unfamiliar to a homology search is a text problem. Rewriting it so it still folds, still assembles, still catalyzes, and still does the specific dangerous thing is a chemistry problem, and the second problem is far harder than the first. Regular readers will recognize this: it is the same gap I wrote about when an AI lab announced new materials and when a model returned a confident structure for a protein that does not behave that way. Prediction is cheap. Existence is expensive.

The follow-up that did the experiment

Someone noticed the same gap, and they were largely the same people.

A preprint titled "Experimental evaluation of AI-driven protein design risks using safe biological proxies" was posted to bioRxiv, with a second version dated 27 January 2026. The author list includes Wittmann and Horvitz from the original team, alongside researchers from the National Institute of Standards and Technology and from a DNA synthesis provider. It states the problem in plain language:

Recently, Wittmann et al. described an in silico pipeline of generative AI tools to reformulate sequences of concern (SOCs) as synthetic homologs that may evade detection by biosecurity screening software (BSS) used by nucleic acid synthesis providers. Experimental testing of synthetic homologs is required to ascertain the true severity of this vulnerability.

"In silico." "Experimental testing is required to ascertain the true severity." The authors of the original paper are saying, in print, that their own study did not establish severity.

So they built a framework using safe biological proxies, harmless stand-in proteins that let you measure whether a redesigned sequence still functions without ever handling anything hazardous, and they ran it. Their finding:

We determine that common AIPD models in use at the time this study was initiated (early 2024) are not yet powerful enough to reliably rewrite the sequence of a given protein, while both maintaining activity and evading detection by BSS.

The models could evade screening. The models could preserve function. Asked to do both to the same protein at the same time, reliably, they could not. Evasion and function pull against each other, which is what a working biologist would have guessed and what nobody had shown.

Note the time stamp the authors put on their own conclusion: "at the time this study was initiated (early 2024)." That is not a claim about 2026 models, and they are careful not to make one. It is also a preprint, which means it has not completed peer review.

Why an EW officer finds this familiar

I spent my Army career on the detection side of exactly this problem, in a different medium.

Counter-IED work is signature matching. You build a library of what threats look like, you deploy sensors that compare the world against the library, and you alarm on a match. The failure mode is permanent and structural: the library describes what you have already seen. An adversary who changes the signature, even trivially, even in ways that do not make the device more effective, walks past a sensor that was correct about everything it knew.

Biosecurity screening is the same architecture. Homology search against a database of sequences of concern is signature matching. A generative model asked to produce a distant homolog is a signature generator. The vulnerability found in the Science paper is not an exotic AI phenomenon, it is the oldest result in detection engineering, arriving in a new medium.

And here is the part I want to defend, because it cuts against the deflationary read of this report. In EW you patch the library before you have proven the new signature belongs to a working device. You do it because the cost asymmetry is enormous: updating a detection rule is cheap and reversible, and being wrong in the other direction is not. The screening providers who shipped those patches made the correct call, and they made it correctly in advance of the wet-lab evidence.

Both things are true at once. The defensive fix was justified before severity was established, and the public threat claim outran the evidence. Those statements only look contradictory if you think evidence standards for "should we patch" and "what should we tell the public is now possible" are the same standard. They are not, and conflating them is how you get both complacency and panic from the same paper.

What I am not claiming

Not that the vulnerability was fake. It was real, it was specific, and three of four screening providers agreed enough to ship code. A screening system that can be walked past by a reformulated sequence is a defective screening system regardless of whether anyone has yet built the protein.

Not that this is settled or safe. The wet-lab result is explicitly scoped to models from early 2024. Capability in this area has not been static, and the honest reading is that a bar was measured, not that a bar was proven permanent. The residual 3% escape rate after patching is also not zero.

Not that the researchers overstated anything. They did the opposite. The hedge is in the abstract, and the same group funded and ran the experiment that tested their own claim. That is unusually good practice and it deserves saying plainly.

Not a technical account of the evasion method. I have deliberately kept this at the level already published by Science, AAAS and Microsoft, and I have left out the pipeline specifics. Nothing here is a recipe and nothing here is intended to be.

And an honest gap: the Science paper is paywalled and I did not read the full text. I read the complete abstract through Europe PMC, the AAAS release, Microsoft's two writeups, and the full abstract of the bioRxiv follow-up through the bioRxiv API. The 75,000-variant figure, the four providers, the three that patched, and the 3% residual all come from the AAAS release rather than from the paper itself, and I have not verified the more precise variant and model counts that circulate in secondary coverage.

The signal

Three things worth carrying.

First, and this generalizes well past biosecurity: when an AI safety result is reported, ask whether anything was built. "The model produced a design that would evade detection" and "the design was made and it worked" are separated by the entire cost of an experiment, and press coverage compresses that gap to nothing. The tell is a phrase like "likely to retain function," "predicted to be active," or "in silico." Those are not hedges journalists added, they are the authors telling you exactly where the evidence stops.

Second, any detection system built on a library of known-bad examples inherits the same weakness, whether it screens DNA orders, network traffic, radio signatures, or academic misconduct. This is not a reason to abandon signature matching, which catches the overwhelming majority of real cases. It is a reason to never treat it as a wall. A screen that has never been red-teamed by a competent generative model has not been tested, it has only been used.

Third, the right evidence bar depends on what the evidence is for. Patch on suspicion, because patching is cheap. Claim on demonstration, because claims move policy and attention. The Science paper and its follow-up together are a clean example of a field getting this right in sequence: fix first, measure second, and publish both. The failure was not in the science. It was in the summary.

The version of this story worth remembering is not "AI can design bioweapons" and not "it was nothing." It is that a real hole in a real choke point was found and closed by people who then went and checked their own homework, and found the threat was, for now, harder to realize than the headline implied. That is what functioning safety research looks like, and it reads as much less dramatic than either thing the internet made of it.