The Signal Report // AI IN THE LAB
AI in the Lab.
Where machine learning genuinely helps research, and where 'AI discovered X' is doing a lot of the lifting. All reports →
17 in this beat · page 1 of 2
Four Labs, One Catalyst, Four Answers
Machine learning for catalyst discovery runs on experimental data, and somebody has to produce that data at a bench. Four laboratories agreed on a protocol, used identical catalyst batches, and ran the same CO2 hydrogenation. Inside each lab the input-output relationships were clear. Pooled across all four they stopped being statistically significant, and the culprit was as mundane as how the reactor was stirred.
Read the report → Report 093 · AI in the Lab · 2026-08-18 · 10 minThe Model Learned the Doctor
A team in Seoul added machine learning to a blood test for a rare adrenal tumor across 20,516 patients, and performance went from good to excellent. Then, while building the app, they found the model had learned which follow-up tests the clinician ordered, which is a record of what the clinician already suspected. They disclosed it themselves, unprompted.
Read the report → Report 087 · AI in the Lab · 2026-08-15 · 9 minYour AI Result Won't Reproduce
Temperature zero is the setting people reach for when they want a model to behave like an instrument. In a study that set temperature to zero, top-p to one, and fixed the seed, one model returned a byte-identical answer on 0.0 percent of questions across ten runs. The largest source of the variation is not in your prompt or your settings: it is how busy the server was.
Read the report → Report 081 · AI in the Lab · 2026-08-12 · 9 minThe AI Read the Spectrum. Now Count the Guesses.
The most-quoted accuracy figure for AI structure elucidation from NMR is 69.6%. Two things about it: that is the rate at which the right molecule appears somewhere in fifteen guesses, and it was measured on spectra a computer generated. On 106 real spectra, the same framework scored 33.0%. Getting structures out of spectra is my day job, so here is what the metric means.
Read the report → Report 075 · AI in the Lab · 2026-08-09 · 9 minWhere AlphaFold Quietly Breaks the Chemistry
A lab made the wrong protein by accident. Measured, it had twice the helix of the original and came in threes instead of ones. Four AI predictors, given that same sequence, handed back the original to within an angstrom, with charged atoms parked in the greasy core where charge is not allowed to sit. Why the wrong answer looked so right.
Read the report → Report 069 · AI in the Lab · 2026-08-06 · 9 minThe AI That Can't Beat a Straight Line
Five single-cell foundation models, the kind sold as a step toward simulating a living cell, were tested against baselines written to be as stupid as possible. One of them predicts that nothing happens. On the task of finding surprising biology, nothing beat it. The useful part is what a baseline is for, and how easy it is to pick one that cannot win.
Read the report → Report 063 · AI in the Lab · 2026-08-03 · 9 minCan an AI Replicate a Research Paper?
Three benchmarks now test whether AI agents can reproduce published research, and all three report a headline score in the low twenties. That looks like independent confirmation. One of them hands the agent the authors' working code; another explicitly forbids it. Underneath sits a fact about how often humans fail the same test.
Read the report → Report 057 · AI in the Lab · 2026-07-31 · 8 minFake Citations in Real Papers
An audit of 2.5 million biomedical papers found references pointing at studies that do not exist have risen more than twelvefold since 2023. The number is smaller than the headlines suggest, and a citation that does not exist is the one kind a machine can catch. The dangerous version is the citation that resolves.
Read the report → Report 051 · AI in the Lab · 2026-07-28 · 7 minDid the Robot Chemist Discover Anything?
In 2023 an autonomous lab reported making 41 novel compounds in 17 days with no human hands. In January 2026 the paper was corrected: the number is now 36, the word "novel" is gone from the title and abstract, and the closing line no longer says "discovery." A spectroscopist reads the diff, and the measurement step where automation broke.
Read the report → Report 046 · AI in the Lab · 2026-07-25 · 7 minAsking Three AIs Is Not a Second Opinion
Six frontier models from two companies, 50 papers, and an open-ended task where they should have scattered. They converged instead. A 2026 position paper measured it, and the deeper problem is what the published literature never taught any of them: the failures nobody writes down.
Read the report →