Report 016 · AI in the Lab
In a chatbot, hallucination is a failure nobody checks. In protein design, it's the actual, published name of a method, and the check is built in. The difference between the two is the whole lesson.
"Hallucination" is the word we use to dismiss AI. The chatbot invents a court case that never happened, cites a paper that doesn't exist, and we call it a hallucination and move on. So it catches people off guard to learn that in one of the most respected corners of AI-for-science, "hallucination" isn't the failure mode. It's the name of the method, chosen on purpose, printed in the title of a 2021 Nature paper from David Baker's lab: "De novo protein design by deep network hallucination." As someone who consults on AI in the lab and spent years doing physical structural measurement, I find this the most honest use of the word in the whole field, and it's worth understanding why.
Start with what a structure-prediction network like AlphaFold does: you feed it a protein sequence, and it predicts the folded shape. Hallucination runs that engine backward. You begin with a random string of amino acids, which folds to nothing in particular, and then you mutate it over and over, keeping the changes that make the network more and more confident it's looking at a clean, well-folded, protein-like structure. You never tell it what to build. You just push until the network is sure it's seeing a real protein. Whatever sequence you land on is the "hallucination," a protein the model dreamed up out of noise because the math said it should hold together.
That is genuinely clever, and when it works it's striking. But notice the trap built into the idea. You've optimized a sequence to make a model confident. Model confidence is not the same thing as a molecule that actually folds in a test tube. And that gap is where the real story lives.
The Baker lab didn't just publish pretty predictions; they made the proteins and tested them, which is the part that earns my respect. Here is their own accounting, verbatim: "We obtained synthetic genes encoding 129 network hallucinated sequences, expressed and purified the proteins in E. coli, and found that 27 folded to monodisperse species with circular dichroism spectra consistent with the hallucinated structures."
Sit with those numbers. They built 129 of the model's confident dreams into real DNA, grew the proteins, and 27 came out as well-behaved, correctly folded molecules. That's about one in five. The other four in five did not clear the bar, even though the network was confident about all of them. And the payoff of the ones that worked was real: the team solved three of the hallucinated structures by hand, two by X-ray crystallography and one by NMR, and those matched the designs closely. So the method genuinely produces novel, foldable proteins. It just produces a lot more misses than the phrase "AI designs proteins" would ever suggest.
Here's the distinction I keep coming back to. When a chatbot hallucinates, the output is confident text, and usually nobody checks it before it does damage. When this network hallucinates, the output is a testable physical claim: this sequence will fold into this shape. You can express it and find out. Four times out of five in that study, the answer came back no, cheaply and unambiguously, at the bench. That's not the AI quietly failing. That's the AI proposing and the laboratory disposing, exactly the way science is supposed to work.
That "circular dichroism spectra consistent with the hallucinated structures" clause is doing quiet, heavy lifting, and it's close to my own history. Circular dichroism is a spectroscopy method: shine polarized light through the protein and the spectrum tells you whether it actually folded into the helices and sheets it was supposed to. A physical measurement, not a model score, is what got a vote on whether each dream was real. The model can generate a million candidates for pennies. The instrument is the referee that decides which ones are proteins.
The field has moved on from this specific 2021 technique. Newer generative tools have improved on it, and design is a hotter area than ever. But the shape of the thing hasn't changed, and it's the same lesson I've written about when an AI "discovered millions of materials" or "found a hidden drug pocket": the model is a proposal generator, and the number that tells you how good it is isn't how confident it was, it's how many of its proposals survived contact with reality. So when you read that an AI designed a protein, ask the two questions that actually matter. How many did it design, and how many of those were made and shown to fold? A method that dreams up a thousand and delivers two hundred real proteins is a triumph, because two hundred novel proteins by hand is a career. But it is still a story about a fast generator paired with a strict physical filter, not a story about a machine that simply knows the answer. Read the verb, and then read the hit rate.
U.S. Army combat veteran (Counter-IED / Electronic Warfare), peer-reviewed researcher in microwave spectroscopy, and founder & CEO of Shroombiosis. Consults on laboratory operations, AI, and supplement formulation.