The AI materials paper MIT disowned

What the paper claimed

The preprint, "Artificial Intelligence, Scientific Discovery, and Product Innovation," was posted to arXiv on 21 December 2024 by Aidan Toner-Rodgers, then a second-year PhD student in MIT's economics department. It was a single-author paper.

Its setup was what every AI-and-science researcher had been waiting for. It described the R&D lab of "a large U.S. firm," working on materials for healthcare, optics and industrial manufacturing, which rolled out an AI materials-discovery tool in three waves starting in May 2022. Teams were said to be randomly assigned to waves, so early and late adopters could be compared like a natural experiment. The tool, trained on existing materials, generated "recipes" for new compounds predicted to have target properties, and scientists then picked candidates to make and test.

The headline results, from the abstract:

• AI-assisted researchers discovered 44% more materials.
• That led to a 39% increase in patent filings and a 17% rise in downstream product innovation.
• The bottom third of scientists saw little benefit, while the output of top researchers nearly doubled.
• AI automated 57% of "idea-generation" tasks.
• 82% of scientists reported reduced satisfaction with their work.

It was a great story with a human twist, and it spread fast. According to TechCrunch, MIT economists Daron Acemoglu and David Autor both praised it, with Autor telling the Wall Street Journal he was "floored." The paper had been submitted to the Quarterly Journal of Economics, but it was never peer reviewed or published.

What MIT said

On 16 May 2025, MIT's economics department posted a statement. It said concerns had been raised about the integrity of the research, that MIT had run an internal, confidential review, and that the paper "should be withdrawn from public discourse." It quoted the letter MIT's Committee on Discipline sent to arXiv:

MIT has no confidence in the provenance, reliability or validity of the data and has no confidence in the veracity of the research contained in the paper.

MIT said student privacy law and its own policy barred it from disclosing the outcome of the review. It said it had asked both arXiv and the journal to withdraw the paper, that arXiv normally only accepts withdrawal requests from authors, and that the author had not submitted one. It also said the author was no longer at MIT.

Acemoglu and Autor, who are thanked in the paper's acknowledgments, added their own statement. They said they had developed concerns over time and brought them to MIT, and that the findings "should not be relied on in academic or public discussions of these topics." TechCrunch, citing the Journal, reported that the concerns started when a computer scientist with materials-science experience approached the two economists in January 2025.

What arXiv did, and what it didn't

On 20 May 2025, arXiv posted a version 2 of the paper. It's a 1 KB placeholder, submitted by "arXiv Admin," with this note: "Withdrawn by arXiv administrators due to concerns about the validity of the data and incomplete Institutional Review Board requirements."

Here's the part most people miss. Withdrawal on arXiv doesn't delete anything. The abstract page shows the version history, and version 1 is still listed and still downloadable. I downloaded the full 78-page v1 PDF while writing this post. So anyone who lands on an old link, a cached copy, or a slide that quotes the abstract gets the original numbers with no warning attached.

What isn't known

It's worth being precise, because the public record is thin. MIT has not said publicly what its review found, only that it has no confidence in the data or the research. The arXiv note adds two reasons, data validity and incomplete Institutional Review Board requirements, without detail. I haven't seen any of the underlying data, and as far as I can tell no one outside the review has. So the defensible statement is not "the data were fabricated." It's that the institution where the work was done, and the two senior economists closest to it, have publicly said the results can't be relied on.

One small discrepancy, for the record: MIT's statement says the preprint was posted "in November 2024," while arXiv's history shows the first submission on 21 December 2024. I'm going with arXiv's timestamp.

The questions a lab person would have asked

I've worked in labs and I consult on bringing AI into them, so here's my honest reaction to the original abstract. None of this is evidence of what MIT found. It's the checklist I'd want anyone to run on a result this clean.

What counts as "discovering a material"? In a real lab, a new compound has to be made, characterized and shown to have the property you wanted. A 44% jump depends entirely on how that was counted, and who did the counting.

Can anyone else look? The firm was never named, and the data came from inside one company's R&D operation. That's common in economics of innovation, but it means nobody can independently check the numbers.

Is the effect too tidy? A randomized rollout across 1,018 scientists at one lab, with clean effects at every stage from discovery to patents to products, plus a survey on job satisfaction, is an enormous dataset for one student to assemble. Not impossible. Worth a second look before it becomes a citation.

I'd add a fourth that applies to this whole beat: an AI-and-science result gets more scrutiny, not less, when it confirms what the field hopes is true. This one told everyone what they wanted to hear about AI, expertise and productivity, which is exactly why it traveled.

How to check a number before you repeat it

This takes about a minute, and it would have caught this case.

Open the abstract page, not the PDF. On arXiv, the "Submission history" box lists every version. A withdrawn paper shows "(withdrawn)" next to the latest version, plus a comment explaining why. A PDF link skips all of that.

Check whether it was ever peer reviewed. A preprint isn't wrong because it's a preprint, but a widely quoted one that never appears in a journal is a flag worth following up.

Search the title with "withdrawn" or "retracted." Institutions and journals post notices, and they're usually easy to find once you look.

Don't lean on an AI assistant for this step. Language models are trained on snapshots of the web, and a paper that was famous before its withdrawal can be remembered as sound. I tested how badly open models handle retractions in Report 159. And if you want the real version of "did AI discover new materials," the published record has its own arguments; see Report 051.

What would change this post: MIT or the author publishing the review's findings, the data being released, or the paper resurfacing in revised form. If any of that happens, I'll update this post in place.