The model learned the doctor, not the disease

This beat usually works one direction: a press release claims more than the paper supports, and the job is to find the gap. Today the gap runs the other way. A professional society put out a release in which the researchers walk back their own headline finding, in their own words, before anyone had questioned it. That is rare enough to be worth explaining properly, and the failure mode they caught is one of the most common ways a laboratory model can look brilliant and be worthless.

What the test does and why it needed help

Pheochromocytomas and paragangliomas, shortened to PPGL, are rare tumors in or near the adrenal glands. They secrete catecholamines, so the first-line screen is a blood test for plasma free metanephrines, the breakdown products of those hormones. The test is sensitive, which is what you want for something that is dangerous to miss. The cost of that sensitivity is false positives, and this is the part clinicians live with: an elevated result frequently means something other than a tumor, including kidney function, other conditions, and a long list of medications that interfere.

Researchers in the department of laboratory medicine and genetics at Samsung Medical Center in Seoul went at that problem with the kind of dataset that makes this worth doing. They pulled 20,516 adults who had undergone metanephrine testing at the center between 2011 and 2024. Among the 19,797 patients who did not have PPGL, 25.2 percent still showed elevated metanephrines. One in four people without the tumor screened positive.

That is a real and well-scoped problem, and machine learning is a reasonable tool to point at it. The idea was to take the metanephrine result and combine it with structured information already sitting in the electronic health record: kidney and urine biomarkers, the patient's medications, the presence of other diseases. All of it plausibly relevant, all of it already collected. The models did what you would hope. Per the society's release, the metanephrine test on its own showed good reliability, and adding the machine-learning layer pushed performance into the excellent range by cutting false positives.

If the story stopped there, it would have been an ordinary conference abstract with an encouraging result, presented as abstract B-091 at ADLM 2026 in Anaheim on 29 July. It did not stop there.

The part that happened after the abstract was submitted

Se-eun Koo, a clinical chemistry fellow and the study's lead author, described the sequence plainly in the release. First, what they thought they had:

Our initial machine-learning models suggested that combining plasma metanephrine results with structured clinical information from the electronic health record could improve real-world discrimination.

Then what happened when they tried to turn it into something a person could use:

After abstract submission, while testing the prototype app with the developed ML model, we realized that some of the apparent improvement might reflect patterns of clinical workup rather than independent biochemical information.

On further analysis, per the release, much of the improvement was explained by shortcut learning from informative missingness. In plainer terms: the model had learned which follow-up tests were ordered. A clinician orders a confirmatory workup when they already suspect PPGL. The model was reading the presence or absence of those orders and, through them, the suspicion of the physician who placed them. It was not extracting a hidden biochemical signal. It was reading the doctor's mind, or more precisely, reading the paper trail the doctor's mind leaves behind.

I want to be exact about what kind of evidence this is, because the beat requires it. This is a conference abstract and a society press release, not a peer-reviewed paper. There is no published methods section to audit, no AUC I can quote from a primary document, and the walk-back is the authors' own account of their own further analysis. That is a limitation on the numbers. It is not a limitation on the lesson, because the failure mode they describe is documented independently and at length, and that literature is where the rest of this report goes.

Both halves of the phrase are technical terms

"Shortcut learning from informative missingness" is not a hedge. It names two specific, separately studied things.

Shortcut learning is the general phenomenon. Geirhos and colleagues gave it its canonical treatment in Nature Machine Intelligence in 2020, defining shortcuts as decision rules that, in their words, "perform well on standard benchmarks but fail to transfer to more challenging testing conditions, such as real-world scenarios." The model is not broken and it is not cheating in any sense it could be blamed for. It found the easiest available rule that separates your training labels. If an easier rule than the biology exists in your data, it will find that one, every time, and your validation set will applaud because the easy rule works there too.

The cleanest medical demonstration I know of is Zech and colleagues in PLOS Medicine in 2018, on pneumonia detection in chest radiographs. They assembled 158,323 chest radiographs across three institutions and found that pneumonia prevalence differed enormously between sites, 34.2 percent at Mount Sinai against 1.2 percent at the NIH cohort and 1.0 percent at Indiana. The consequence is the single most useful number in the AI-in-medicine literature:

The prevalence of pneumonia was high enough at MSH (34.2%) relative to NIH and IU (1.2% and 1.0%) that merely sorting by hospital system achieved an AUC of 0.861 (95% CI 0.855–0.866) on the joint MSH-NIH dataset.

An AUC of 0.861 for detecting pneumonia, achieved by detecting which hospital took the picture. A model that learned nothing but the scanner's fingerprint would post a respectable number and go in a press release. Same structure as the metanephrine case, different substrate: in Seoul the giveaway was the order history, in New York it was the equipment.

Informative missingness is the specific mechanism, and it has its own literature. In clinical data, whether a measurement exists is rarely random. Somebody decided to order it. That decision encodes clinical judgment, so the presence or absence of a value carries information beyond the value itself. Rolf Groenwold laid out the consequences in Diagnostic and Prognostic Research in 2020, under a title that is the whole argument: "the curse of knowing."

His point is subtler than "missingness is a confounder," and it is the reason I think this deserves a full report rather than a line in a roundup. A model built on informative missingness can be genuinely predictive at the moment you build it, and then degrade for a reason that has nothing to do with drift in the patients:

A feedback loop arises, when informative patterns in the data influence measurement practices that subsequently change the information that is captured by particular (missing) data patterns.

Groenwold's worked example is cholesterol in cardiovascular risk. If a GP did not order a cholesterol test, that absence is favorable information, because it means the GP was not worried. Build the model on that. Now deploy it, and tell clinicians that the model exists. The moment ordering behavior responds to the model, the missingness pattern that carried the signal is gone, and it was destroyed by the model's own success. His conclusion is that researchers building such a model "should anticipate the missing data mechanism once the model is deployed in clinical practice," which is a design requirement, not a caveat.

Why the lab is the worst place for this and the best place to catch it

I work on laboratory operations and on AI systems that sit next to instruments, so let me say what makes this specific to the lab rather than a generic data-science warning.

A laboratory information system is a near-perfect record of clinical suspicion. Every order is a decision made by a human being who had a reason. The panel ordered, the time it was ordered, whether it was repeated, whether a confirmatory test followed, how fast: all of that is a high-resolution transcript of what the ordering clinician was thinking. Then it is stored in exactly the tidy, structured, machine-readable form that makes it trivial for a model to consume. We built the most leak-prone possible feature set and we built it on purpose, for good reasons that have nothing to do with modeling.

So when a lab model outperforms the assay it was meant to improve, the first hypothesis should not be that it found new chemistry. The first hypothesis should be that it found the ordering clinician. The prior is not neutral here. It leans hard toward leakage, and the burden should sit with the model.

The flip side is that laboratory medicine is unusually well equipped to catch this, because the field already has the concept of a reference method. Lab people are trained to ask what a number is traceable to. The audit that catches shortcut learning is the same instinct applied to a model: what is this prediction traceable to, and would it survive if that thing were removed?

What an actual audit looks like

Koo's own framing of the takeaway is the right one, and it is more useful than most conclusions that survive peer review:

The key message is not that machine learning cannot help, but that routine-care models must be audited to confirm they are learning the intended clinical signal.

Concretely, four things, and none require exotic tooling.

Ablate the process features and see what is left. Retrain with every trace of clinical workflow removed: which tests were ordered, when, in what sequence, and the missingness indicators that stand in for all of it. Keep only the measured biochemistry. Whatever performance survives is the part you can attribute to biology. If the excellent result collapses back to the assay's baseline, you have your answer, and you have it before it reaches a patient.

Check whether missingness alone is predictive. Train a deliberately stupid model on nothing but the pattern of which values are present. This is the metanephrine equivalent of Zech's sort-by-hospital test, and it costs an afternoon. If that model performs well, the ceiling on your real model's biological content just dropped, and you know by how much.

Validate somewhere the ordering culture is different. External validation is standard advice and usually justified by case mix. For this failure mode the reason is sharper: ordering habits are institutional. A different hospital has different protocols, different specialists, different thresholds for confirmatory testing. A model leaning on workflow will fall over there specifically, and that is the point of running it.

Ask what happens after deployment. This is the Groenwold question and the one most easily skipped, because it is about a future state rather than a held-out set. If the model is used, does the behavior that generated its signal change? A model whose input is clinician suspicion, deployed to inform clinician suspicion, is wired into its own feedback loop. It can degrade precisely because people started using it.

The part I would keep

Strip away the machine learning and there is still a finding here worth having: in 20,516 patients at a single large center over thirteen years, one in four people without PPGL screened positive on the first-line test. That is a solid, useful number about a real clinical burden, and it did not depend on any model.

And the disclosure itself is the story. The authors had a good result, an accepted abstract, and a conference slot. Nobody had challenged them. They found the flaw while building the app, which is to say they found it at the moment the work stopped being a number and started being a thing someone would use, and then they told the society's press office in language specific enough to be checked. The self-correction cost them their headline. It is also the only reason I trust anything else in the abstract.

The version of this that worries me is not the honest one in Anaheim. It is the same model, in a lab that never built the app, presented as an AI system that improves rare-tumor detection. There is nothing in the accuracy metrics that would have exposed it. Performance was excellent. It measured the right outcome on held-out data from the same institution, which is what most validation means in practice. It would have passed.

That is the durable lesson, and it is why the phrase to carry out of this is Koo's: confirm the model is learning the intended clinical signal. Accuracy will not tell you. Accuracy is the thing that gets fooled.