← The Signal Report Work with me

The Signal Report // AI IN THE LAB

AI in the Lab.

Where machine learning genuinely helps research, and where 'AI discovered X' is doing a lot of the lifting. All reports →

43 in this beat · page 1 of 5

Report 255 · AI in the Lab · 2026-11-19 · 7 min

What the scientific method buys an AI agent

An AI agent built to hypothesize, predict, test and reconcile wrote a DNA-motif algorithm that beat a standard tool. Then its author preregistered a teardown. Seven of ten loop variants failed to beat blind sampling from the same model, the full scaffold scored below it at 7 to 8 times the tokens, and a bare prompt won. The control every AI-scientist claim should face.

Read the report →
Report 249 · AI in the Lab · 2026-11-15 · 7 min

What the AI chemist's yield gain measures

OpenAI says GPT-5.4, wired to Molecule.one's robotic lab, proposed TEMPO as an additive for a stubborn drug-chemistry coupling, lifting mean yield from 16.6% to 25.2% over 10,080 reactions. The yields are detector-model estimates, the best bench numbers come from hand-picked pairs, TEMPO was first tried in 1998, and the chemistry preprint never mentions the AI.

Read the report →
Report 243 · AI in the Lab · 2026-11-10 · 7 min

Why AI scientists fail, in six patterns

A January 2026 preprint reported four attempts to let AI agents take machine-learning research from idea to paper. Three failed; one earned a borderline acceptance. The six ways the agents went wrong, from quietly simplifying the method to announcing success on broken results, make a checklist for any "AI did the science" headline.

Read the report →
Report 237 · AI in the Lab · 2026-11-07 · 6 min

The AI materials paper MIT disowned

A preprint said an AI tool helped 1,018 materials scientists find 44% more materials and file 39% more patents. MIT said it had no confidence in the data and arXiv withdrew it, but version 1 still downloads. What is and isn't known, and a one-minute check before you repeat a number.

Read the report →
Report 231 · AI in the Lab · 2026-11-04 · 7 min

Why AI rarely predicts "no effect"

Thousands of scientists graded AI hypotheses built from their own papers. Coverage said no AI proposes null hypotheses; the chart says rarely, and humans rarely do it either. The bigger finding: AI judges barely agreed with the experts.

Read the report →
Report 225 · AI in the Lab · 2026-11-01 · 6 min

What Kosmos's 79.4% accuracy measures

The AI scientist's headline number is an average of 102 statements. Data analysis scored 85.5%, but interpretation hit 57.9% on a test that only asked whether it followed logically. And the six-month claim came from polling seven beta users.

Read the report →
Report 219 · AI in the Lab · 2026-10-29 · 7 min

What an AI paper passing peer review measured

Nature published the paper behind Sakana's AI Scientist, and the line that traveled was that an AI-written paper passed peer review. One of three did, at a workshop that accepts 70%, after humans chose which to submit. And the AI reviewer that grades the rest of the paper accepts about half of what humans reject.

Read the report →
Report 213 · AI in the Lab · 2026-10-26 · 7 min

What data leakage does to an AI accuracy claim

An AI accuracy number is only as good as its test set. Kapoor and Narayanan found data leakage, the test answers sneaking into training, documented in 294 papers across 17 fields. In civil war prediction, the only four papers claiming complex models beat logistic regression all had it; fixed, the edge vanished. Eight ways it happens, and five questions that catch it.

Read the report →
Report 207 · AI in the Lab · 2026-10-23 · 7 min

What made rentosertib an "AI drug"

Rentosertib, now in a 320-patient phase 3 for lung fibrosis, is the drug people cite when they say AI can discover medicines. The two papers behind it show AI ranking a target under human-set filters, generating chemistry people then optimized, and a phase 2a trial whose main endpoint was safety. The fine print on the +98.4 mL result.

Read the report →
Report 200 · AI in the Lab · 2026-10-19 · 7 min

What 722 AI math papers actually prove

OpenAI posted 722 AI-written math papers to GitHub, then withdrew three, including a Hodge conjecture claim, over one sign error. None of the three had a Lean-checked proof. The 22% and 42% formalized figures both trace to the repository, and they count different things.

Read the report →