83%: The accuracy score Microsoft’s AI diagnostic system claims for complex cases

The gap between artificial intelligence and human medical expertise narrowed significantly on Thursday, at least on paper. Microsoft released a study showing its new diagnostic system, MAI-DxO, correctly identified complex medical conditions in a simulation 83% of the time. That figure stands in stark contrast to the 36% accuracy rate recorded by primary care doctors and general internists working individually with standard reference materials. While the disparity is eye-catching, the comparison carries significant caveats regarding who exactly the AI was up against.

The system operates as a coordinated panel of AI agents that mimic a roundtable of specialists. These agents question patients, order tests, and debate diagnoses before settling on a final answer. Microsoft AI CEO Mustafa Suleyman framed the result as a critical step toward what he calls medical superintelligence. He described the technology as foundational infrastructure designed to create real breakthroughs in how healthcare is delivered, moving beyond simple data retrieval to active clinical reasoning.

However, the study’s own data suggests that the raw power of the underlying language models drove most of the improvement, with Microsoft’s orchestration layer adding incremental gains on top. The team tested the system with models from several major competitors, including OpenAI, Google, Anthropic, Meta, and DeepSeek. The best performance came from OpenAI’s GPT-5 and GPT-5.5 models. This reliance on third-party intelligence raises questions about where the true value lies within Microsoft’s stack.

Cost and complexity play a major role in these results. When using GPT-5 with a basic prompt, the system achieved 78.9% accuracy at an estimated cost of $8,400 per case. Adding the DxO orchestration reduced that cost by approximately $3,000 while improving accuracy by only four points. The benefits were even less pronounced in emergency department simulations, where both the standalone model and the orchestrated system performed similarly in terms of cost and accuracy.

The advantage becomes more pronounced with weaker models. Gemini 2.5 Flash, for instance, scored 46.4% on its own but jumped to nearly 69% when assisted by DxO. Despite this boost, the overall gap between stronger and weaker models remained larger than the specific improvement provided by the orchestration system. Microsoft researchers noted that accuracy gains naturally diminish as models approach theoretical maximums, positioning DxO as a tool for optimizing both cost and precision rather than a standalone miracle cure.

One critical caveat in the published data is the control group. The complex cases used in the simulation were drawn from the New England Journal of Medicine. In a real hospital, these scenarios are typically handled by specialists or physician teams, who were not part of the comparison. Comparing an AI panel to solo generalist doctors may overstate the performance gap that clinicians would actually face in a collaborative setting.

Microsoft is not alone in making these claims. The company joins Google and OpenAI in asserting that its AI can match or outperform human doctors. Google recently published a similar clinical study with Beth Israel in The Lancet, evaluating its AMIE diagnostic AI in real-world urgent care visits. Microsoft aims to go further by partnering with the Mayo Clinic to train a dedicated healthcare foundation model, with a release planned for next year. Suleyman expects this model to allow for enormous leaps in capability.

To validate these claims beyond simulations, Microsoft has partnered with Beth Israel Deaconess Medical Center in Boston. This new evaluation will test DxO’s clinical reasoning across various hospital settings, including emergency rooms and urgent care clinics. Patients will receive standard treatment while consenting participants also interact with the AI system in parallel. Microsoft and its partners will then compare the AI’s recommendations against those made by human doctors. Dominic King, vice president of Health at Microsoft AI, called this the biggest study of its kind, noting it will provide compelling evidence on the system's scalability.

Suleyman, a former co-founder of DeepMind, believes the technology is ready for real-world stress tests. He predicts that healthcare models will improve by an order of magnitude in capability within the next 24 to 36 months. He compares the current moment to early advances in coding and chatbots, suggesting that once a production flywheel begins, progress accelerates rapidly. Microsoft already sells dictation and radiology tools to clinicians and views DxO as a major expansion of that portfolio, aiming to become the leader in clinical decision support in the coming years.