Microsoft says its AI diagnoses illness more accurately than doctors

Microsoft says it has built an AI tool that diagnosed medical conditions up to four times more accurately than human doctors in testing. The results were published Thursday in Science, a major research journal. There is a big catch. The tool has only been tried on past cases, so nobody knows yet how it would do with real patients.

What the tests showed

Microsoft pitted its system against physicians on cases drawn from medical case reports and emergency department data. On cases from hundreds of reports in The New England Journal of Medicine, the AI was right about 83 percent of the time. Generalist doctors scored about 36 percent with reference materials and 22 percent without them.

On emergency room cases from Beth Israel Deaconess Medical Center in Boston, the AI scored 80 percent. Doctors averaged 47 percent. The AI also tended to reach its answers at a lower estimated cost.

How the tool works

Microsoft calls it the Diagnostic Orchestrator. It uses several AI agents, which are separate AI programs that work together like a panel of experts. They reason through symptoms and medical history, decide which tests to order, and settle on a diagnosis. The goal is to copy how doctors investigate a case. The system runs on OpenAI's GPT-5.

Microsoft built its own test set, called SDBench, and plans to make it open source, meaning others can use it. The peer-reviewed paper does not include newer AI models, so it is not a comparison against the latest rivals.

The catch

Microsoft itself says the tool has only been tested on past medical cases. How well it would perform in a real clinic is unclear. Mustafa Suleyman, who leads Microsoft AI, said it needs clinical validation. He called that a leap, but described the results as a major milestone.

The doctors in these tests were also working from case files, not sitting with a patient. That is a different situation from daily medical care.

What happens next

Dr. Dominic King, Microsoft's AI health lead, says testing in hospital settings with Beth Israel Deaconess is likely to get underway in the next couple of months. Neither he nor Suleyman gave a firm timeline. Microsoft has not said when or how the tool might be sold. King stressed that medicine has strict regulatory and compliance rules that would need to be followed.

Suleyman predicts impressive deployments of AI systems next year. He says human clinicians will always have a central role, and hopes AI will take some of the drudgery out of medicine so doctors can spend more time with patients.