A recent NPR News article by Joseph Kim tells the story of AI Consult, a clinical decision support tool powered by OpenAI’s GPT-4o, that helped identify a congenital heart defect in a 4-month-old boy in Nairobi, Kenya. Clinical officer Vyonne Njeri initially believed the baby’s fever and blocked nose pointed to a routine cold. While documenting the visit, a yellow alert on her screen flagged the infant’s elevated heart rate. She examined the child again with her stethoscope and detected a heart murmur. “That’s something I would have missed on any other day,” she told Kim. The baby was referred to a specialist, who confirmed the diagnosis and started treatment.
AI Consult fits into a clinician’s normal workflow without interrupting it. As clinicians write their notes, the system reviews the information in real time and displays one of three alerts: green when everything appears normal, yellow when something deserves a closer look, and red when urgent attention may be needed. Researchers evaluated the tool in a cluster-randomized trial across 16 Penda Health clinics in Nairobi and Kiambu counties. The study included nearly 10,000 patient visits and 103 clinical officers, with about half using AI Consult. The study involved nearly 10,000 patient visits and 103 clinical officers, with about half using AI Consult. The results published in Nature Medicine in June, showed the system cost about four cents per patient. Njeri, who often sees five or six patients an hour without immediate access to a specialist, said the AI’s recommendations are helpful about half the time. Even when they do not change her decisions, she said they are rarely wrong.
Even so, the study found no measurable improvement in patient outcomes. Treatment failure within 14 days occurred in 2.2 percent of patients treated with AI assistance and 2.0 percent of those treated without it. After adjusting for differences between clinicians and clinics, the gap was not statistically significant. The researchers concluded that the tool appeared safe, but any direct benefit for patients is likely small. One finding did stand out. An independent panel of six Kenyan family physicians found that clinicians using the AI produced higher-quality medical records, including more accurate diagnoses, more complete documentation, and stronger treatment plans.
Dr. Bilal Mateen, the study’s co-author and chief AI officer at the nonprofit PATH, which sponsored the trial, told Kim that treatment failures are uncommon in primary care, making them difficult to measure in studies of this size. He estimated that detecting a meaningful difference would require a trial involving about 139,000 patients. Dr. Jonathan Chen of Stanford, who was not involved in the research, praised the study for evaluating AI in routine clinical practice instead of controlled simulations. Dr. Nicholas Okumu, a surgeon at Kenyatta National Hospital who also was not involved in the study, urged caution. He said regulatory approval alone does not guarantee safety and stressed the need for ongoing oversight. The Gates Foundation funded the trial and also supports NPR’s global health coverage.
Related Posts
Kim points out that the study raises an important unanswered question. Clinicians valued the tool, documentation improved, and the cost of using it was minimal. Yet those strengths did not translate into better patient outcomes, suggesting that proving such benefits in primary care may be extremely difficult. Better clinical records are valuable in their own right because they support more consistent and informed care. Still, as AI becomes more common in healthcare, especially in resource-constrained settings, evidence that these systems improve patient outcomes should come from the tools themselves rather than relying on the judgment of the clinicians who use them.
Acknowledgement: This summary was generated using AI writing tools as part of my experimental learning process. For more information, check out the original article listed below.