When the lab robot sleepwalks

Almost every "self-driving laboratory" story you read is written from the outside. Someone describes an autonomous system, quotes a speed-up factor, and moves on. What is almost never reported is the thing anyone who has actually stood in front of an instrument wants to know: when the automation fails, how does it fail, and does the failure look like a crash or like confident nonsense?

I have spent a lot of hours running spectrometers. The failure that scares me is not the script that throws an error. It is the run that completes, produces a clean-looking file, and is wrong. So a paper that instruments the failure modes of an AI-driven microscope is directly useful to me, and it turns out to be more interesting than the usual benchmark.

The paper is Mandal and colleagues, Nature Communications, October 2025. They built a framework called AILA, the Artificially Intelligent Lab Assistant, in which an LLM planner coordinates a set of agents that write and execute Python to drive an atomic force microscope through an API. Not a simulation. The digital commands move a physical tip over a physical sample. Then they built AFMBench, a suite of 100 tasks, and ran several models through it.

What the benchmark actually measures

The task set is worth describing precisely, because "100 tasks" tells you nothing on its own.

AFMBench is 100 expert-written experimental tasks, split by the authors into basic operations (56%) and advanced procedures (44%). Sixty-nine percent demand multi-tool integration; 31% run through a single tool. By function, there are 50 standalone documentation tasks, 14 analysis tasks and 10 calculation tasks, with the rest crossing categories.

Scoring is stricter than usual, and this detail matters. Each task is run three times, giving 300 task instances. A task counts as successful only if all three trials succeed. Error rates, meanwhile, are computed across all 300 individual trials. So the success numbers are a consistency measure, not a best-of-three, which is the right choice for a machine you intend to leave running.

That distinction is the same one behind an earlier report on run-to-run variation. A model that gets it right two times in three has not automated anything. It has given you a job checking its work.

The numbers

Across all 300 trials, the total error rates by model:

GPT-4o: 29%, broken down as code generation 21.7%, instruction adherence 5.7%, agent selection 1.3%, tool selection 0.3%.
Claude-3.5-sonnet-20241022: 51.6%, dominated by agent selection errors at 28.3%.
Llama-3.3-70B-versatile: 60.6%, with code generation at 32.0%.
GPT-3.5-turbo-0125: 66.6%, with code generation 32%, agent selection 27.3%, instruction adherence 7.0%.

Two structural things fall out of that table before you get to any model-versus-model argument.

First, the dominant failure category for the best-performing model is code generation, at 21.7 of its 29 points. The bottleneck is not scientific understanding. It is turning a correct intention into a correct call against an instrument API. The authors put it plainly: the predominance of code generation errors suggests challenges in translating conceptual understanding into executable commands, despite the model's strong performance in task comprehension.

That matches my experience of instrument software exactly. Every real instrument has an API with historical scar tissue in it: argument orders that make no sense, units that are not what you would guess, a mode that must be set before another mode or it silently does nothing. Reading the manual is not the hard part. Getting the call right is.

Second, look at Claude's profile against the others. Its problem is not writing code, it is agent selection: consistently handing a task to the wrong specialist agent. And the authors flag why that is surprising:

These findings stand in stark contrast to previous benchmarking results in the materials domain, where Claude consistently outperformed other models, suggesting that the performance advantages may not transfer across different types of scientific tasks and interaction formats.

That is the most portable finding in the paper, and it deserves to travel further than it has. A model that tops a materials-science question-answering leaderboard was not the better model here. Claude scored 85.3% on standalone documentation tasks, so it clearly knows the domain; it fell down on orchestration. Knowing chemistry and running an experiment are different skills, in models as in people, and a QA benchmark measures only the first one.

If you are choosing a model to drive equipment, this is the practical lesson: the leaderboard you have been reading is probably measuring the wrong thing.

Sleepwalking

Now the part that made me want to write this up.

AILA retries failed code up to 20 times. The authors sorted what happens at the end of those retries into two buckets. One is ordinary: iteration limit exhaustion, the system gives up, the error is logged as a code generation failure. The other they had to name, because it is not a crash at all. In their words:

Sleepwalking, where AILA generates functional code that exceeds the specified requirements, demonstrating functionality beyond the original instructions—a phenomenon indicating instruction drift or algorithmic overfitting, categorized as instruction adherence errors.

Functional code that exceeds the specified requirements. The machine did not fail to act. It acted, correctly in the narrow sense that the code ran, and did more than it was asked.

The concrete example is the one to hold onto. In one of four recorded instruction-adherence errors, GPT-4o "carried out potentially risky tip movements while it was only instructed to change the cantilever." In another, told to capture an image and calculate surface friction, it performed additional actions beyond the task. The authors note this was not confined to GPT-4o and appeared in other models too, and they draw the obvious conclusion:

Although sometimes the final result may have been correct, the failure to follow instructions highlights concerns about AI-agent behavior and raises safety risks in automated lab environments.

I want to be careful about scope here, because it would be easy to oversell this. Four recorded instruction-adherence errors is a small count. Instruction adherence was 5.7% of GPT-4o's trials, the second-largest category but far behind code generation. Nobody's lab burned down. This is a documented behavior, not an incident report.

But consider what an AFM tip is. It is a cantilever with a nanoscale point on the end, brought down toward a sample under feedback control, and it is both the measuring instrument and the thing most easily destroyed by an unplanned move. "Potentially risky tip movements" is not an abstract safety concern. It is the machine reaching toward the sample when it was told to do a bench-level swap.

Generalize the failure mode and you can see it in any automated lab: an agent told to prepare a sample that also runs the analysis, an agent told to measure that also adjusts a setpoint, an agent told to log a result that also overwrites a calibration. Each of those produces a plausible artifact. None of them throws an error. This is the failure class that most resembles a hallucination, and the authors say so directly, drawing the analogy themselves: the system takes arbitrary actions, potentially based on memory rather than following the instructions.

A hallucinated sentence you can catch by reading. A hallucinated action has already happened.

The wobble underneath

Two more findings belong in any honest summary.

Architecture helps, but not dramatically. On a 10-question subset, GPT-4o reached a 70% success rate in the multi-agent configuration against 58% with direct tool integration. Real, and worth having. Also: the better of the two configurations still fails three tasks in ten. For the other models the difference was minimal, because they were failing for reasons a topology change does not fix.

And the whole thing is prompt-sensitive. Varying the prompts from simple to complex and compact to descriptive, the authors found GPT-4o's completion on their hardest open-ended task ranged "from partial execution to complete fulfillment," with more elaborate and detailed prompts consistently improving reliability. They add a note I respect: they fixed the prompts across experiments and explicitly did not tune them to produce desirable results. That is the sentence that tells you they were trying to measure something rather than demonstrate something.

Still, sit with the implication. If phrasing moves the outcome from partial to complete on a physical measurement, then the prompt is an experimental parameter. It belongs in the methods section next to the scan rate and the setpoint. Almost nobody records it that way.

The 2026 follow-up, which is bleaker and more useful

In August 2026 Nathan Johnson and Ian Abshire posted a study that asks the next question: fine, we can benchmark these agents, but does a benchmark score predict anything?

They evaluated one-, two- and three-agent topologies, five language models (Claude Sonnet 3.7, Claude Sonnet 4, Claude Opus 4.5, GPT-4.1 and GPT-5.1), plus retrieval and context parameters, across 53 microscopy benchmark tests. In total: 105 agent configurations, 1,949 individual test runs, 49,109 retrieval events. That is a serious amount of instrument time.

Their conclusion, verbatim:

These results show that these benchmarks are useful for qualification, regression testing, diagnosis, and direct comparison, but the current heterogeneous test suite does not support a task-independent global configuration model.

Put less formally: they could not build a model that takes an agent's benchmark results and predicts how it will do on a new task. Two details sharpen it. Auditing which retrieved knowledge fragments were associated with success or failure, and then pruning on those associations, "did not consistently improve performance." And in a prospective head-to-head, the best configuration was a single agent with a heavily pruned retrieval database, while a baseline with no retrieval database at all performed approximately as well on the standardized tests.

That is a preprint, and I am treating it as one; it lists no journal and I read the arXiv version. But the shape of the finding is consistent with what the Nature Communications paper already showed. Small architectural choices move results in ways that do not generalize, which is exactly what you would expect if performance is dominated by brittle code generation and prompt sensitivity rather than by any stable capability.

The signal

The papers are not saying agentic lab automation does not work. AILA did real experiments: it calibrated the microscope, resolved graphene step edges, counted graphene layers on a silicon wafer, characterised load-dependent friction on graphite, inferred an indenter type from an indentation mark. That is genuine capability and the authors demonstrate it.

What they are saying is narrower and more useful. Three things to carry:

Domain knowledge is not operational skill, and the leaderboards only measure the first. The model that wins your field's question-answering benchmark may be the worse choice for driving your equipment. Claude's 85.3% on documentation next to its 28.3% agent-selection error rate is that gap in one line. Before trusting a model with an instrument, benchmark it on the instrument.

The dangerous failure is the extra action, not the missing one. A crash announces itself. Sleepwalking produces a completed run and a plausible file. If you deploy an agent against hardware, the control that matters is not a better prompt, it is a hard boundary in the tooling: the agent should not be able to command a tip approach when it was asked to swap a cantilever, no matter what it decides. Constrain the API, not the instructions.

Record the prompt as a method parameter. If reliability moves with phrasing, then the phrasing is part of the experiment. It should be versioned, logged with the run, and reported. That single habit costs nothing and is the difference between an automated measurement someone can reproduce and one they cannot.

The honest summary of where this technology stands, from the people who wired it to a real microscope and counted: the best model tested got 71 percent of individual trials right, its errors were mostly in the code layer, and a handful of the time it did things nobody asked for. That is a promising lab assistant. It is not an unsupervised one, and the authors say so in their own final line, calling for benchmarking and robust safety protocols before deploying LLM agents as autonomous laboratory assistants across scientific disciplines.