Microsoft's MAI-DxO scores 83% on NEJM cases; doctors 36%

Microsoft's diagnostic system cleared peer review today. The interesting part is not the multiple in the headline — it is which doctors you compare it to, and how much of the win belongs to the model underneath rather than to Microsoft's orchestration.

Microsoft's MAI-DxO answered 83% of 304 complex case records from the New England Journal of Medicine correctly, against 36% for primary care doctors who had textbooks and the internet, according to results published in Science on Thursday. Doctors working without those reference materials scored 22%. That makes the system 2.3 times more accurate than a doctor with references, and 3.7 times more accurate than one without them.

Expect to see "four times better than doctors" in a lot of coverage this week. The paper does not support four. Its own best multiple is 3.7, and it only gets there by comparing a multi-agent system against a single generalist with the books taken away. That distinction is the whole story, because the second number — 2.3x — is the one a hospital administrator would actually weigh.

What the benchmark does differently

The team built a test called SDBench rather than reusing medical licensing exams, and the reasoning is sound. USMLE-style questions hand over every fact upfront and ask for one answer. A real diagnosis is sequential: you ask something, you get an answer, you decide what to order next. SDBench runs both halves of that — 304 NEJM clinicopathological conference cases, the ones physicians argue about at case conferences because they were misdiagnosed in life, plus a set of more ordinary emergency-department presentations drawn from MIMIC-IV-ED. Every case is interactive. The scoring covers the diagnosis, the evidence requested and the estimated cost of the tests.

MAI-DxO is not a model. It is a coordinator that runs a set of agents as a virtual physician panel: each proposes a diagnosis, picks the next question or test, and challenges the others, over several rounds of debate, with the cost of each test weighed against how much it would narrow the differential. Microsoft's own write-up reports 80% accuracy on the emergency-department cases against 47% for physicians, and a cost-tuned configuration that held 81% on the NEJM set at less than half the cost of the comparison systems.

The orchestration is a real result. The cost result may be the more useful one, since the bottleneck in clinical AI is rarely "can it name the disease" and usually "how many scans did it order to get there."

The model does most of the work

Here is where the paper gets less flattering, and where the coverage will get sloppy. Microsoft tested its coordinator on top of several model families — OpenAI, Google, Anthropic, Meta and DeepSeek — and its best numbers came from OpenAI's GPT-5 and GPT-5.5. TechCrunch's read of the same accuracy-and-cost data found that DxO adds roughly four points on top of GPT-5 alone on complex cases while cutting about $3,000 from a per-case cost of roughly $8,400. With GPT-5.5, the lift was larger on paper, from 72% to nearly 87%, at a slightly lower cost. On the emergency-department cases, DxO changed almost nothing: about 85% either way, at similar cost.

The gap between weak and strong models is bigger than the gap DxO creates. Gemini 2.5 Flash scored 46.4% on its own and nearly 69% with the coordinator — a 22-point gift, and still 14 points behind what GPT-5 managed with no coordinator at all. That is the honest shape of the finding: orchestration helps most when the underlying model is weaker, and the frontier models are already close to the ceiling the benchmark allows.

The other caveat sits in the study design. The human comparison was generalist physicians working individually, and cases this hard are normally handed to specialists or a physician team. Nobody measured the specialist. Simulation is also not care: no case here involved a patient who could deteriorate, a family asking questions, or a record that was missing the one lab that mattered.

Why Microsoft is publishing this now

The Science paper lands with an agenda attached. Mustafa Suleyman, who runs Microsoft AI, called the result a step toward "medical superintelligence," which Microsoft defines as surpassing the strongest human-led health systems across the full arc of care rather than winning a benchmark. The company is working with Mayo Clinic on a healthcare foundation model it says will ship next year, and it has signed Beth Israel Deaconess Medical Center in Boston for a prospective study where consenting patients interact with DxO in parallel with standard care and the recommendations are compared afterwards. Google took a similar route with its AMIE system, running a prospective study with Beth Israel that it published in the Lancet this month.

Microsoft already sells dictation and radiology tools into hospitals, and Suleyman was blunt about the commercial intent: the company needs to own the best clinical decision support product in the market within a few years. A peer-reviewed accuracy number is the entry ticket to that conversation, which is exactly why the doctor baseline in the paper deserves the scrutiny it will get.

US banks are blocking AI agents that log into customer accounts with the customer's own credentials, after finding third-party agents inside accounts and, in some cases, taking actions that appeared independent of the user's instructions, Axios reported. Morgan Stanley has shut agent access off entirely, and JPMorgan, Bank of America and Citi are restricting it, according to The Decoder's account of the reporting. Most banks are steering toward read-only access: an agent may look at balances, not move money.

The uncomfortable detail is that banks cannot reliably tell an authorized agent from a rogue one, because both arrive with valid customer credentials entered by a human who was told to log in. That is not a model-capability problem, it is an identity problem — the agent inherits a session that says "this is the customer." Banks are rewriting their terms of service around that gap, and any agent vendor that wants to survive the next year will need delegated authorization that a bank can verify, scope and revoke. Until then, read-only is the compromise: keep the agent in the loop, keep it away from the money.

What to watch: whether the Beth Israel prospective study reports a real-world accuracy gap or a benchmark-only one, and whether any US bank publishes an agent-access policy with an actual verification standard in it rather than a block.

Would you let an AI agent read your bank balance if the bank could prove it was acting on your instructions? Write to us via our contact page.

Kyunghyun Cho launches Ortet, a frontier AI health lab, with $500M

A co-author of the attention mechanism spent five years building AI for drug discovery inside Genentech. Today he launched his own lab — with a half-billion dollars behind it, one disclosed backer, and no priced round. Ortet launched on September 29 as a frontier AI lab for health, with a $500 million commitment from Thoreau and a founding team led by Kyunghyun Cho.

A 243-cell 'periodic table' puts every AI at zero on control

The most useful thing to come out of a paper this week is a map, not a result. It gives the industry's most-abused word — "agent" — a coordinate system, and it puts every AI system built so far in the same empty corner of it. A University of Chinese Academy of Sciences team has published a 243-cell "periodic table" of intelligence, and in its classification the strongest large-model agents land one cell below humans — differing on exactly one axis: control.

Peking University's 591-run RL study finds uniform sampling wins

A Peking University group built the comparison platform, ran the experiments, and reported the boring answer: with everything else held constant, the data policy is not what moves the score. The sharper finding is that the benchmark summary you choose can flip which method looks best. Peking University's Data-Centric AI group ran 591 reinforcement-learning experiments to test whether cleverer data policies beat plain uniform sampling, and could not find one that reliably does.

AI forecasters underestimated progress on every benchmark that resolved

The people best placed to call AI's next milestone keep calling it too late. An audit of five years of expert forecasts puts a number on the gap, and shows the error pointing the same way almost every time. On four AI benchmarks that have already resolved, the domain experts surveyed by the Forecasting Research Institute had assigned an average probability of 24.6 percent to the outcomes that actually happened.

Black Forest Labs' 7B robot model beats models twice its size

Black Forest Labs took its video model into robotics and put it on top of NVIDIA's robot benchmark — with a model small enough to run on the machine itself. Also today: Databricks buys its way into spreadsheets, and India's Sarvam claims a document-AI record. Black Forest Labs released FLUX 3 Action, an open world-action model for robots that tops NVIDIA's RoboLab-120 leaderboard at 42.9% success with only seven billion parameters.

NIO's MM-Future plans driving by imagining 64 futures at once

A NIO paper turns consequence-aware planning into a benchmark score, Waymo stacks 271 million driverless miles behind its safety claim, and the AlphaFold Database quietly became pandemic infrastructure. Ren Shaoqing's name is on an autonomous-driving paper for the first time in a decade, and the method is the one the world-model field has been circling: let the car imagine many futures and pick the one it likes best.

Bristol researchers: medicine already handles AI's black boxes

Two Bristol philosophers argue the medical AI field is optimizing the wrong thing about black boxes. Pharma ran into the same wall decades ago and built a process that works without ever explaining the mechanism. Emanuele Ratti and Lena Zuchowski, both at the University of Bristol, argue in a paper accepted by Studies in the History and Philosophy of Science that machine learning should stop treating explainability as the gate to trust and instead certify models the way medicine certifies…

Bilibili opens a 100-model arena — GPT-6 Astra takes the top slot

Two stories where the ranking comes from outside the lab: China's biggest video platform just turned creator reviews into a live model leaderboard, and a clinical voice company published the first benchmark of whether AI can actually say a drug's name. Bilibili's new AI Infinite Arena ranks models by how often they win — not by how well they score.