Report 159 · AI in the Lab
A researcher took 161 of the most publicised retracted papers in science, handed their titles and abstracts to three open-weight language models, and asked a single question: has this been retracted. The best model got 18 percent of them. Then he checked the opposite direction, fed the same models 34,070 ordinary published papers, and found something stranger than the first result.
If you run a lab, or write papers, or just read them for a living, there is a reasonable chance you have started routing literature through a language model. Summarise this. Is this claim supported. What does the field say about X.
There is a question underneath all of those which almost nobody asks out loud: does the thing answering know which of the papers it is drawing on have been withdrawn from the scientific record?
Mike Thelwall, of the School of Information, Journalism and Communication at the University of Sheffield, has now tested that directly, and the answer is close to no. The important part is the qualifier he puts on it himself, which most of the coverage has dropped.
The design is deliberately generous to the models. Thelwall took a set of retracted papers that had been recycled from an earlier study: articles flagged by Retraction Watch, ranked by Altmetric attention, the 250 loudest, filtered down to 217 with abstracts. From those he used the 161 that were retracted before October 2023.
That cut-off is the point. Every model he tested has a training cut-off at least nine months later than October 2023, so each of them had the chance to read about these retractions many times over during training. These are not obscure withdrawals. They are the ones that made news.
Three models, all run locally as open weights: GPT OSS 120B, Gemma 3 27B, and DeepSeek R1 70B. The prompt is short and unambiguous. It tells the model it is an academic researcher concerned with whether publications have been retracted, defines retraction as withdrawal from the publishing journal because of issues discovered after publication, and asks for a y/n answer, an explanation, and a copy of the retraction notice if relevant.
Results, from the paper's Table 1, out of 161 papers every one of which had been retracted:
Or, stated the way the abstract states it: in over 80 percent of cases the models said a retracted article had not been retracted.
Two details in the failure mode matter more than the headline percentage.
First, when a model did correctly say "retracted," the reason it gave was usually wrong. Thelwall reports that the journal name was often wrong, the volume number almost always wrong, and the explanation for the retraction usually partly or completely wrong. His worked example is one of the most famous retractions of the decade, the near-ambient superconductivity paper, where Gemma 3 explained the retraction as data fabrication and reproducibility concerns. The actual Nature notice says something different: that the paper does not accurately reflect the provenance of the investigated materials, the experimental measurements undertaken and the data-processing protocols applied, and that concerns were independently raised about the reliability of the electrical resistance data. A correct verdict reached by inventing the reasoning is not a correct answer. It is a coin landing the right way up.
Second, and this is the part I would put on a lab whiteboard, look at how the models justified a "no." Thelwall reports the typical phrasings:
The provided text does not indicate the article has been retracted.
and
There is no evidence or indication from the provided title and abstract that it has been withdrawn or retracted.
He draws the obvious conclusion, and it is the right one: the model was scanning the text it had just been handed for the word "retracted" rather than consulting anything it learned in training. You asked it a question about the world. It answered a question about the paragraph.
That distinction is invisible in the output. Both a genuine recall of the retraction record and a quick scan of the supplied abstract produce the same confident sentence.
The second half of the study is the benchmark: does the model cry wolf, flagging valid papers as retracted? Thelwall obtained FTP access to MDPI's complete journal set in XML, selected eight journals across fields, capped each at a random 5,000 articles for a benchmark set of 34,070 papers, and ran the same prompt over both full texts and title-plus-abstract.
On this axis the models do well. Across the roughly 49,000 full-text responses that came back in a usable format there were 55 false retraction claims, an error rate of about 0.11 percent, and 28 false claims on the title-and-abstract runs. Three genuinely retracted papers had slipped into the benchmark set by accident, with "RETRACTED:" at the start of their titles, and the models caught eight of nine of those, which tells you they can read a label perfectly well.
But when they did cry wolf, they did it with extraordinary confidence. Thelwall reproduces one fabricated output in full: a complete, formatted retraction notice for a paper that had never been retracted, naming a journal that was not the paper's journal, inventing a DOI, citing three specific prior publications as the sources of supposed plagiarism, invoking COPE guidelines, and signing off from an editor-in-chief. None of it happened.
Then he asked the follow-up question that turns this from a curiosity into something operational. If the models are not detecting retractions from memory, are they detecting problems in the papers themselves? He ran a second prompt asking each model whether the article contained any clear errors in the design or application of its research methods, instructing it not to mention errors it was unsure about.
Gemma 3 27B found clear methods errors in 97 to 100 percent of articles in every one of the eight journals. GPT OSS 120B found them in over 90 percent of articles in six of the eight. A manual spot-check of 30 of those claims found them sometimes wrong, sometimes technically correct but trivial (an "increasing" where "decreasing" belonged, a missing decimal point), and plausible in two cases that were too complex to adjudicate.
Put the two results side by side, because together they say more than either does alone. The model cannot tell you which papers the community has formally withdrawn. The same model will tell you that essentially every paper you show it is methodologically broken. It has no calibrated sense of scientific reliability in either direction: it under-detects the failures that actually happened and over-detects failures that did not.
Headlines on this line of work tend to land on "AI can't spot retracted papers," full stop. Thelwall's own conclusion is narrower and he states it explicitly:
This applies only to the open weights locally run versions since the web interfaces can run web searches to check directly.
That is a real and material limit, and it should change what you take away. If you paste a title into a consumer chat interface with search enabled, it may well find the retraction notice, because Retraction Watch and the publishers put those notices on the open web and the tool is reading the web rather than its own weights. The failure documented here is a failure of parametric knowledge, not of the whole category of tool.
Which makes it a failure with a very specific address: it lands on locally hosted, air-gapped, or API-only deployments with retrieval turned off. That is exactly the configuration a lot of labs and companies choose, for good reasons, when they are pointing a model at unpublished data or protected information. If your institution runs an offline model over your literature because the data cannot leave the building, this study is about your setup and not about the chatbot on your phone.
There is separate evidence that hosted tools are not automatically safe either. In a 2025 study, Thelwall and colleagues asked ChatGPT 4o-mini to evaluate the quality of 217 retracted or otherwise concerning studies, thirty times each. In the authors' words:
Surprisingly, none of its 6510 reports mentioned that the articles were retracted or had relevant errors, and it gave 190 relatively high scores (world leading, internationally excellent, or close).
A follow-up in the same paper extracted 61 claims from retracted articles and asked whether each was true, ten times each. It gave a definitive yes or a positive response two-thirds of the time, including for at least one statement that had been shown to be false over a decade earlier. Separately, a group led by Konradin Metze at the State University of Campinas tested 21 chatbots against the retracted work of Joachim Boldt and, as reported by Retraction Watch, found they identified fewer than half of the retracted papers on average while falsely flagging around 18 percent of Boldt's valid papers. I was not able to open that paper myself, so I am relaying its reported figures rather than confirming them.
My own research is in microwave spectroscopy, which has nothing to do with bibliometrics. But the shape of this problem is completely familiar, and it is the same one I wrote about in Report 87 and Report 57.
A retraction is not a property of a paper's text. You cannot find it by reading the paper more carefully, because the paper does not contain it. It is a property of a separate record, held elsewhere, updated after the fact, by people. A system whose only access to the world is the statistical residue of a training corpus has no reliable route to a fact of that kind, and no way to know that it lacks one.
This is the same reason a spectrometer cannot tell you your sample was mislabelled upstream. The instrument answers the question it was built to answer, with total composure, on whatever you put in front of it. Everything about whether that answer means anything lives outside the instrument.
The main study is a preprint and has not been peer reviewed. I read arXiv:2604.16872v1, posted 18 April 2026, in full as a PDF, including both tables. It is a single-author paper and its numbers are his. I have not seen a published version and I did not attempt to reproduce anything.
There is a discrepancy inside the paper. Its abstract names the third model "DeepSeek R1 72B" while the methods section and Table 1 both say 70B. I have used 70B, which is the standard release size and the figure the tables carry, and I note the abstract's number rather than quietly correcting it.
A large share of the benchmark runs returned nothing usable. The paper reports that in 46,968 of the full-text cases no answer was given or the response format was non-standard, with DeepSeek in particular often ignoring the requested format. The false-positive rate is therefore computed over the cases that did return a parseable answer, and I have not tried to estimate what the unparsed ones would have said.
I read the 2025 Learned Publishing paper as its publisher-deposited abstract, not in full. The article is open access under CC BY, but both the publisher's site and the institutional repository copy sat behind bot challenges I could not pass. The quoted passage is verbatim from the abstract Wiley deposited with Crossref. I did not read its methods or results.
I did not open the Metze et al. paper. The publisher's site blocked access. I verified its existence and full citation through Crossref and PubMed records, and every number I attribute to it comes from Retraction Watch's report, which I did read, and is labelled as such in the text above.
These results are specific to three models, one prompt and one language. Thelwall says so himself in his limitations. Other models or prompts could do better, and none of this is a statement about tools with live retrieval.
The practical version of this is short.
If you use a language model for literature work, treat "has this been retracted" as a question it structurally cannot answer from memory, and check the retraction record separately. That means Retraction Watch's database, the publisher's own page for the article, or a reference manager that syncs retraction flags. This is a lookup, not a judgement call, and it takes seconds.
If you run an offline or retrieval-free deployment, understand that you have opted out of the one mechanism that makes any of these tools reasonable on this question. That may still be the correct trade for your data, but it is a trade, and it should be written down somewhere rather than assumed away.
And if a model volunteers that a paper has a clear methodological error, weigh that against the finding that one of these models said the same of 97 to 100 percent of everything it was shown. A critic who objects to every paper is not a strict reviewer. A critic who objects to every paper is a broken instrument, and the fact that it phrases its objections fluently is precisely what makes it worth distrusting.
Scope note: this report describes what three locally run open-weight models did on one prompt in one preprint, plus two peer-reviewed studies of hosted chatbots. It is not a benchmark of any current commercial product, it does not test tools with live web retrieval, and nothing here should be read as a claim about how a specific named assistant behaves today. The principal source is a preprint that has not been peer reviewed, one supporting paper was read as its publisher-deposited abstract only, and one was not opened at all and is cited through secondary coverage, as itemised above.
U.S. Army combat veteran (Counter-IED / Electronic Warfare), peer-reviewed researcher in microwave spectroscopy, and founder & CEO of Shroombiosis. Consults on laboratory operations, AI, and supplement formulation.