Terminal-Bench-Science: the best AI agents solve only 30% of real research tasks

Share
Terminal-Bench-Science: the best AI agents solve only 30% of real research tasks

A new benchmark built from researchers' own lab work is drawing the first honest line between AI agents that can talk about science and agents that can actually do it. The answer is humbling: the best system in the world clears fewer than a third of the tasks.

Terminal-Bench-Science 0.1 launches today with 70 tasks drawn from real scientific workflows across the life, physical, Earth, mathematical, and engineering sciences. Built by the Harbor framework team with Stanford's AI lab and domain experts, it extends the Terminal-Bench methodology that has driven progress in software-engineering agents into scientific research. Scientists — not model developers or data vendors — wrote and reviewed the tasks, which span data analysis, statistical inference, simulation, optimization, theorem proving, image reconstruction, and scientific machine learning.

The headline number is deliberately sobering. Every model ran three independent trials on each task, and Claude Opus 5 with Claude Code came out on top by resolving just 30.0% of them. GPT-5.6 Sol with Codex followed at 22.4%, and Claude Fable 5 at 21.4%. Claude Opus 4.8 sits mid-pack at 10.5%, while GPT-5.6 Terra, Kimi K3, and Grok 4.6 all clear less than 10%. The strongest open model is GLM 5.3 at 8.1%, and GPT-5.6 Luna brings up the rear at 3.3%.

The gap between this benchmark and its sibling is the point. Compared with Terminal-Bench 3.0, resolution rates drop more than ten percentage points for every model evaluated on both — and the authors say that's deliberate: tasks were calibrated to challenge the newest frontier models, so the room for progress is real and measurable rather than already saturated. Cost matters too. GPT-5.6 Sol matches Fable 5's performance at under a third of the cost ($4.2k versus $14.2k across the full suite), while Opus 5 reaches the highest resolution at $7.0k. Only Kimi K3 and Opus 5 land on both the cost and token-efficiency Pareto frontiers.

The takeaway here isn't that AI is failing at science — it's that "useful research assistant" is still a much harder bar than "writes good code," and we finally have a yardstick that says exactly how far the frontier has to move. A benchmark that pulls real, verifiable workflows from working scientists, and refreshes as the frontier advances, is the kind of instrument the field has been missing. Expect it to become a standard citation whenever a lab claims its model is a scientist.

Would you trust an AI agent that clears less than a third of your lab's real workflows? Write to us via our contact page.

Sources: Terminal-Bench-Science (GitHub)

Get SiliconNoon in your inbox

One email a day with the AI stories that matter — what happened, the numbers, and why it matters. No spam, unsubscribe in one click.

Read more

Fudan's Whitzard agent ranks No. 2 on CyberGym exploit benchmark

Fudan's Whitzard agent ranks No. 2 on CyberGym exploit benchmark

A Chinese university team just proved that an open-weight model plus strong agent scaffolding can out-hack the labs that built the models. Fudan University's Whitzard agent jumped to No. 2 globally — and No. 1 among universities — on CyberGym, the AI security benchmark, cracking 91.2% of the real-world vulnerabilities it faced. The new leaderboard, released this week, puts the team led by Professor Yang Min's lab in the world's first tier of AI attack-and-defense research.

A 243-cell 'periodic table' puts every AI at zero on control

A 243-cell 'periodic table' puts every AI at zero on control

The most useful thing to come out of a paper this week is a map, not a result. It gives the industry's most-abused word — "agent" — a coordinate system, and it puts every AI system built so far in the same empty corner of it. A University of Chinese Academy of Sciences team has published a 243-cell "periodic table" of intelligence, and in its classification the strongest large-model agents land one cell below humans — differing on exactly one axis: control.

DeepMind alumni's Inherent says its Faraday agent beats GPT-5.5 at reproducing research

DeepMind alumni's Inherent says its Faraday agent beats GPT-5.5 at reproducing research

A London lab founded by Google DeepMind alumni says its new agent, Faraday, outperformed Claude Opus 4.8 and GPT-5.5 at independently reproducing the findings of published scientific papers — and it did it on a 27B-parameter Qwen 3.6 model, a fraction of the frontier systems' size. Inherent emerged from stealth just weeks ago with a $50 million seed round, and this benchmark claim is its first real proof point. The company is upfront that the task was narrow: replication, not discovery.

DeepSeek's V4-Flash-Vision rivals Opus 4.8 on agent tests

DeepSeek's V4-Flash-Vision rivals Opus 4.8 on agent tests

DeepSeek keeps moving fast on its low-cost track, and its latest move pairs image understanding with the company's signature price. Today it unveiled V4-Flash-Vision Exp, an experimental multimodal model that adds vision to its V4 Flash series, and DeepSeek says it stacks up against Anthropic's flagship on visual-agent benchmarks. DeepSeek's V4-Flash-Vision Exp nearly matches Opus 4.8 on multimodal agent tests. The new model, live on DeepSeek's API platform as deepseek-v4-flash-vision-exp…

Shanda founder's 27B local model takes the agent fight to DeepSeek

Shanda founder's 27B local model takes the agent fight to DeepSeek

StartLux's first preview beats much bigger models on agent benchmarks, VAST ships production-ready 3D scene generation from a single image, and Hugging Face gives coding agents a memory the user controls. Shanda co-founder Chen Danian's StartLux unveiled a 27-billion-parameter local agent model on Wednesday that, by its own numbers, runs within striking distance of DeepSeek's trillion-parameter flagship on tool-using tasks.

Humanoid robot scores first autonomous full table-tennis match at Beijing conference

Humanoid robot scores first autonomous full table-tennis match at Beijing conference

The 2026 World Robot Conference opened in Beijing on Monday with a milestone that puts embodied AI on the radar of anyone who follows the field: a humanoid robot completed the world's first fully autonomous table-tennis match — not a scripted rally, but a real game with serving, receiving, and adaptive shot selection against a human opponent. 超维动力 KAI (Chaoweidongli), a full-stack embodied AI company, demonstrated its SMASH 2.0 system on a standard ping-pong table at the Beijing Yichuang…