Microsoft had a big week in medical AI. MAI-DxO landed in Science beating physicians at diagnosis, Microsoft AI mapped its plans for medical superintelligence, and the same team published a framework for what that would actually take.
There’s a long road ahead. The Science paper formalizes SDBench, which turns 304 NEJM case conferences and 325 MIMIC-IV emergency admissions into step-by-step encounters where the diagnostician asks questions, orders priced tests, and decides when to commit.
MAI-DxO plays a panel of virtual physicians that debate each case.
- On NEJM cases, DxO hit 83% versus 36% for physicians with references (22% without), and stayed near 80% at a quarter of the doctors’ testing spend in budget mode. On ED cases: 80% to 47%.
- It asked 25.5 questions before its first test. Physicians asked 5.6.
- The comparison is against individual generalists in a text-only simulation with no benign cases, no imaging, and no specialists or care teams.
Microsoft is up for the challenge. A companion post frames the paper as a step toward medical superintelligence, which Mustafa Suleyman told Reuters last fall was two to three years away.
- Microsoft is open-sourcing SDBench for select NEJM cases, Gatekeeper, Judge, and cost code included, so others can test their models against it.
- A prospective study at Beth Israel Deaconess will have patients interact with the system directly, measured against physicians’ independent assessments. That’s the same BIDMC that just ran Google’s AMIE study.
- Microsoft is also folding its health scientists and the Dragon Copilot and PowerScribe teams into Microsoft AI, alongside its Mayo Clinic frontier model collaboration.
Now the fine print, also written by Microsoft. The BMJ Digital Health & AI paper defines medical superintelligence as surpassing the strongest human-led health systems across the full arc of care, then lists six dimensions it needs:
- medical knowledge, diagnostic reasoning and triage, therapeutic decisions, longitudinal context, clinical coordination, and alignment with human values.
MAI-DxO covers roughly one of the six. The other five involve treating patients over years, coordinating across teams, and eliciting what patients actually want.
The Takeaway
Give Microsoft credit for publishing its own to-do list alongside the victory lap. Diagnosing NEJM puzzles in a text box is one of six boxes, and the other five cover the parts of medicine that don’t come with an answer key.
