Report 027 · AI in the Lab
AI has quietly taken over parts of the weather forecast, and on an ordinary day it's as good as the physics. Then a 2026 study looked at the days that actually matter, the record-breakers, and found the machine flinching exactly when you need it most.
Over the last three years a genuinely impressive thing happened in weather forecasting. Machine-learning models from the biggest names in AI (Google DeepMind's GraphCast, Huawei's Pangu-Weather, a Shanghai team's FuXi) learned to predict tomorrow's weather by training on decades of past weather, and they got good. On the standard scorecards they now match, and sometimes beat, the physics-based workhorse that national weather services have leaned on for decades: the European Centre's high-resolution model, HRES. They do it in seconds on a single machine instead of hours on a supercomputer. That is a real advance, and I want to say so clearly before I complicate it, because the complication is the whole point of this piece.
A study published in Science Advances in May 2026, led by researchers at the Karlsruhe Institute of Technology and the University of Geneva, asked a sharper question than "how good is the AI on average?" They asked: how good is it on the days that break records? The heat wave hotter than any on file, the cold snap deeper than the archive has ever seen, the wind beyond the charts. Those are the forecasts that matter most, the ones early-warning systems and disaster response actually hang on. And that is exactly where the AI models came up short.
The pattern was consistent across all three AI models. In the authors' summary, "AI models generally underestimate the intensity of heat, cold, and wind records," and here is the part worth reading twice: "The greater the exceedance of the record of their training data, the larger the underestimation." The further past the historical record a real event went, the more the AI lowballed it. During heat waves specifically, the models "consistently predicted temperatures much lower than those observed." The machine didn't just miss. It missed in one direction, always pulling the extreme back toward something more normal.
The physics model didn't share that flinch. HRES tracked the record-breakers far more faithfully. One of the co-authors, Erich Fischer of ETH Zurich, called the result a "warning shot" against swapping out traditional models too quickly. That's the finding in a sentence: on the calm middle of the distribution, AI is now competitive; out at the violent edges, physics still wins.
This is where my own training kicks in. I'm a spectroscopist by research background, which means I spend my life on the difference between a model that fits data and a model that understands the system underneath it. An AI weather model is the first kind. It is, at its core, a spectacularly sophisticated pattern-matcher: it has seen a huge library of past atmospheres and learned the statistical shape of how one hour of weather turns into the next. Within the range of what it has seen, that is powerful. Ask it about a Tuesday that looks broadly like thousands of Tuesdays in its training data, and it will interpolate a very good answer.
A record-breaking event is, by definition, the one thing it has never seen. And neural networks are notoriously bad at extrapolating past the edge of their training data. Faced with conditions outside its experience, the model does the safe statistical thing: it hedges back toward the average it knows. The research team put it plainly: the AI models "were dealing with events outside their training data and were trying to pull their forecasts back toward more typical historical averages." That instinct, regression toward the familiar, is a virtue almost every day and a liability on the exact day a town needs to know the flood will be worse than any flood before it.
The physics model works from the other end entirely. HRES doesn't ask what past weather looked like; it integrates the governing equations of the atmosphere, the fluid dynamics and thermodynamics that don't care whether a temperature has ever been recorded before. As the researchers put it, "because they never change, physics-based models can better simulate scenarios the world has never seen before." A law of physics extrapolates for free. A fitted curve does not. That is the difference between learning the rules and memorizing the games.
Two of them, because this cuts both ways. First, the AI models are not bad forecasters, and the paper never says they are. They are excellent, fast, and cheap over the vast majority of ordinary weather, which is most of what a forecast is. The lead author, Zhongwei Zhang, noted the models were built for "general weather conditions," with extremes treated as "a secondary task." They weren't designed to fail here; they simply weren't optimized for the tails. Second, nobody serious is arguing to throw them out. The researchers' own recommendation is a hybrid: combine "the speed of AI with the strong foundation of the fundamental laws of physics," and keep verifying before either is trusted alone for "high-stakes applications such as early warning systems and disaster management." The right takeaway isn't AI-bad. It's AI-shaped: know where the tool is strong and where it quietly isn't.
An AI model is a mirror of its training data. It is at its most confident and its most dangerous in the same place: an input near the center of everything it has already seen. A record-breaker sits outside that center by definition, and that is precisely when the machine's answer bends back toward the ordinary. So when you're handed an AI forecast, or an AI anything, the question to carry is whether the case in front of you lives inside the model's experience or outside it. This is the same pattern that runs through this whole beat. An AI that "discovered millions of materials" was confident about compounds nobody had made; an AI that designed proteins was confident about structures that mostly didn't fold; AI-designed drugs ace the early trial and stall where real biology begins. The model proposes, fluently, from what it has seen. Reality keeps its most important events just past the edge of that library, and physics, not pattern-matching, is what reaches them.
U.S. Army combat veteran (Counter-IED / Electronic Warfare), peer-reviewed researcher in microwave spectroscopy, and founder & CEO of Shroombiosis. Consults on laboratory operations, AI, and supplement formulation.