Artificial intelligence is now competing with — and beating — human doctors in diagnosing complex illnesses. Microsoft's newly announced Diagnostic Orchestrator clearly outperformed specialist physicians on difficult medical cases, according to a study published in the journal Science. The research shows the technology does not rely on a single model: the system uses a multi-agent setup that divides tasks the way a panel of physicians would.
In this architecture, agents review the patient's history and complaints, decide which tests to order, and work through a diagnostic process that mirrors the steps clinicians follow.
The numbers leave human doctors far behind
Evaluation was carried out on SDBench, an open-source benchmark assembled from hundreds of case reports selected from The New England Journal of Medicine (NEJM) archive and from emergency department records at Beth Israel Deaconess Medical Center in Boston.
On the NEJM cases, the system achieved accuracy of close to 83%. On the same cases, general practitioners with access to reference sources managed only 36%, while physicians without references reached just 22%. The picture was similar for the emergency department cases: the AI hit 80%, compared with an average of 47% for the physicians.
Teamwork, not raw model power, made the difference
The system's engine is OpenAI's GPT-5 model. The project was initially run on GPT-4.1, and the jump in model capacity could not eclipse the coordination advantage offered by the multi-agent architecture. While the research paper does not include newer models such as GPT-6 Astra, Meta Muse Wonder Full, Google Gemini 4 Argon or Anthropic Opus 5.5, the Diagnostic Orchestrator still delivered better results than every model working on its own.
Clinical trials come before widespread use
Despite the striking figures, Microsoft executives and Microsoft AI CEO Mustafa Suleyman noted that a long road lies ahead before the technology enters routine clinical practice. Because the research relied solely on retrospective case records, it remains unknown how the system would behave in a live hospital setting.
Microsoft plans to launch clinical validation trials to test the technology's performance in real-world conditions within the next few months.
Comments (0)
No comments yet. Be the first!