Best model scores 12% on SemiAnalysis's new frontier benchmark
Every few months a new leaderboard arrives claiming the models have finally been measured on real work. This one arrives with a number nobody can spin: the best model in the world scores 12%.
SemiAnalysis released GDPx on Sunday, a 206-task benchmark split across 11 domains it considers the places frontier intelligence will actually earn its trillion dollars — chip design, hardware design, cybersecurity, biology, finance, legal, quant trading, recursive self-improvement, materials science, alignment and software engineering. Claude Opus 5.5 leads the all-domain table at roughly 12%, winning quant trading, hardware design and chip design outright; Gemini 4 Carbon is the best biology model and edges software engineering; GPT-6 Astra wins recursive self-improvement and materials science convincingly. Finance and legal are the floor: under binary grading, "basically every model scores exactly 0," which is why SemiAnalysis reports partial credit for finance instead.
The design choices are where the benchmark earns its claim. Default tasks run with an eight-hour limit, models run in their native harness — Claude Code for Opus, Codex for OpenAI's models, Antigravity for Gemini, OpenCode as the fallback — and the headline score is pass@1 across three attempts with every domain weighted equally. Fourteen specialist data partners built the tasks: Arithmetic on cyber, Latch Bio on biology, Living Context on finance, Chakra on legal, Third Moment on quant trading, Extrapolate on RSI, Beautiful Machines on hardware design, ValenceForge and Emulated on materials, Fleet on alignment, Datacurve on software engineering, and Aria, Arcverex and Ergodex on chip design. Fireworks supplied inference for the open models, Modal the sandboxes. The full board, with per-domain and cost-versus-performance views, is public at gdpx.semianalysis.com.
The reason the numbers are low is deliberate, and it is the part of the report most worth reading. SemiAnalysis's complaint is about environments as much as difficulty: OpenAI's GDPval, published in September 2025 and still the broad benchmark reported on release cards, hands the model a prompt and a PDF and asks for a deliverable. GDPx drops the agent inside a workplace instead. Its finance tasks sit in a simulated deal world seeded from a real transaction — a private equity fund buying about 62% of a family snack manufacturer for $880 million plus an earnout — spread over roughly 4,000 anonymised files across four deal folders, and the agent has to work out which of them matter before it can do anything. Cyber tasks come from real zero-days with live services running in the environment. Materials science tasks include battery cyclers, a catalyst reactor and a transmission electron microscope. Hardware design runs Siemens NX sessions and tests output on real cloud FPGA instances.
That is a harder thing to score, and it shows in the critique. SemiAnalysis argues GDPval's rubric graders were noisy — human experts agreed with each other only 71% of the time on the gold subset, and the automated grader did worse — with vague criteria, incorrect reference solutions and tasks that rotted as links and prices went stale. Its own QA process, it says, flags nearly 50% of Terminal Bench 4.0's tasks as broken, a pointed claim given how widely that benchmark gets cited.
The second finding will be more awkward for the labs than the 12%. SemiAnalysis says Opus 5.5 and GPT-6 Astra had their overall scores "sandbagged" by safety classifiers that fire on cyber, biology and alignment tasks; strip those three domains and the ranking moves. Its conclusion is that the best cyber model available today is not a frontier flagship at all but Muse Wonder Full, because it refuses less. Any capability comparison between models with different refusal policies is quietly comparing policy as much as skill, and that distinction vanishes the moment the number is printed on a launch slide.
The per-domain spread is its own story. SemiAnalysis notes "meaningful spikiness" across the leading models and, more oddly, similar spikes within a single model family, which is what you would expect from task sets that reward one narrow habit rather than general competence. On the cost-versus-performance frontier the shape matches how these models are already being bought: the cheaper open-weight models fight over the low end, Opus takes the high end, and OpenAI's mid-tier models sit in the middle — with Gemini 4 Carbon flagged as the one to watch once it leaves preview.
Why it matters: a benchmark is a purchasing tool as much as a scoreboard. Teams picking a model for chip layout, contract review or threat hunting need the top line to reflect the work they actually do, and a benchmark built around one-shot document generation stopped answering that question a while ago. GDPx also puts a number on something labs rarely volunteer — that the finance and legal work most likely to be automated next is the work current models fail at completely under strict grading.
Two caveats worth holding. GDPx is a paid SemiAnalysis product, and its authors have an interest in being the benchmark of record; the partner-built tasks are not independently audited from outside. And a 12% top score reflects tasks chosen to be hard, not proof that models cannot do useful work: SemiAnalysis itself notes Opus 5.5 and GPT-6 Astra are "pushing the boundaries of human knowledge." What the number does establish is that the honest frontier figure is well below the release-day charts, and that the gap between a benchmark and a job is now the interesting part.
What to watch: whether labs start reporting GDPx alongside GDPval on model cards, and whether the task set leaks into the next round of training data — the two ways a hard benchmark quietly stops being hard.
Should benchmark scores decide which model your company buys? Write to us via our contact page.
