Everyone's Racing to Build a Smarter Model. The Data Says That Isn't the Bottleneck.

Give a frontier coding agent a real problem lifted from a peer-reviewed Nature-family paper, and it beats the published result on 17.8% of tasks. Hand the identical model a set of pre-built domain skills instead, and completion on a biomedical benchmark jumps from 57.1% to 100%. Same model. Same weights. No bigger model, no more parameters, only better tools.

Those two numbers came out of separate labs, nine days apart, studying unrelated problems. Neither team was arguing about model scale. But put them side by side with two more data points from the same ten days — a rare-disease diagnosis agent already running in production, and the field's biggest lab shipping its own research workbench — and a specific, testable claim falls out: the thing separating a demo from a working AI scientist right now is not reasoning capability. It's whether the agent has the right tools, wired into the right data, running long enough to matter.

The 17.8% problem

NatureBench(arXiv 2606.24530), published June 23 by a 17-author team, is a deliberately hard test. Instead of a synthetic leaderboard, it pulled 90 tasks directly out of real, peer-reviewed Nature-family papers and built an automated harness — "NatureGym" — to grade an agent's attempt against the paper's own published result. Ten agent configurations were run against it. The best one exceeded the original paper's result on 17.8% of tasks under the benchmark's effect-size threshold.

The interesting part isn't the headline number — it's the failure analysis. The paper attributes most losses to "wrong method choice and insufficient compute budget," and separately notes that when agents do succeed, they tend to win by reframing the problem as a conventional machine-learning task rather than by producing genuine scientific insight. In other words: the model can reason. It just doesn't reliably know which method to reach for, or have the compute/tooling to execute the right one once it picks it.

What 57.1% → 100% actually proves

NVIDIA's BioNeMo Agent Toolkit (June 23) is close to a controlled experiment for the same question. The reference agent — GPT-5.5 fast, run through Codex CLI — is held constant. The only variable is whether it has access to "BioNeMo Skills": protein-structure prediction, molecular generation, docking, and sequence analysis, packaged as MCP-documented tools the agent can discover and call, either against hosted NIM endpoints or a local NIM deployment. Without the skills, task completion on the toolkit's life-science benchmark was 57.1%. With them, it was 100%, and the toolkit also delivered "2x improvement in number of passing assertions per tokens consumed" — the agent got both more correct and cheaper per correct answer.

That is the NatureBench failure mode, fixed. "Wrong method choice" stops happening when the right method is a callable tool instead of something the model has to derive from first principles. A minimal version of the pattern looks like this:

# without a domain skill: the agent has to *derive* the right call
result = agent.run("predict whether this point mutation destabilizes the protein")
# -> often reaches for a generic method, or the wrong one entirely

# with a domain skill: the right method is a discoverable, typed tool
from bionemo_skills import predict_structure, score_stability

structure = predict_structure(sequence)
delta_ddg = score_stability(structure, mutation="A123V")
# -> the agent calls a validated tool instead of reinventing structural biology

Where this already works in production

Microsoft Research's Talos (June 24) is the tooling thesis running at clinical scale, not benchmark scale. It's an open-source agent that doesn't answer a one-off prompt — it continuously reanalyzes genomic data for undiagnosed rare-disease patients as new evidence enters the literature, and flags actionable variants for expert review. Across 29 monthly cycles and 4,735 individuals, it surfaced 241 new diagnoses in 238 patients — a 5.1% additional diagnostic yield on top of standard-of-care analysis — with a median 32 days from a new piece of published evidence to a flagged diagnosis. On its validation cohorts it recovered 90% and 87% of in-scope diagnoses; 59% of the new gene-disease diagnoses it surfaced weren't yet curated in OMIM at all.

Talos isn't a general chatbot pointed at a genome. It's a narrow agent, deeply wired into a specific pipeline, run continuously rather than once. That's the same shape as the BioNeMo result — capability comes from integration and persistence, not from swapping in a smarter base model.

Even the biggest lab is betting on plumbing, not model size

On June 30, Anthropic shipped Claude Science, a beta workbench for scientists running on a local kernel. The same day, Anthropic also announced Claude Sonnet 5, a genuinely more capable general model. But Claude Science's pitch isn't "now running on our newest model" — it's "over 60 curated skills and connectors" pre-configured for genomics, single-cell, proteomics, structural biology, and cheminformatics, plus a local kernel and reproducible, auditable output. The field's best-resourced lab, with every incentive to sell "bigger model, better science," chose to sell integration instead.

The counter-evidence: raw scale still stalls

Two other papers from the same window cut against a simpler story — that this is just "add more tools and you're done." Agon(arXiv 2606.24177, June 23) is an autonomous research orchestrator built around a "Zero-Code" principle — it ran 444 iterations of its "Prompt Economy" loop across multiple disciplines from nothing but a topic prompt. Its own framing is "machine scales, human steers": judgment calls are explicitly routed back to a human, not automated away. And NatureBench's authors found that even when agents technically succeed, they tend to do it by reframing a hard scientific question as a conventional ML task — a shortcut, not insight. More tools and more iterations extend what an agent can execute; they don't yet substitute for a scientist deciding what's worth executing.

What this means if you're building or buying an AI scientist

Put together, the pattern across NatureBench, BioNeMo, Talos, Agon, and Claude Science is consistent enough to act on: the marginal unit of capability in this field right now comes from tool integration and continuity, not from a bigger base model. If you're evaluating an AI-for-science product, the question worth asking isn't "whose model is biggest" — it's "how deeply is this wired into my actual data, my actual instruments, my actual pipeline, and does it keep working after the demo ends."

That's the bet behind MegaBrain Science's design: a live kernel the agent shares with you, running on your own machine against your own data, with an independent reviewer that re-derives results before you trust them. Not a smarter chatbot — better plumbing.