Tag: nature

Integrating Consumer AI Into Healthcare Pathways

Posted on by Jason Barry

If AI recommends the right care and nobody is around to get it, did it really recommend the right care? A new perspective piece from Topol and friends in Nature Health doesn’t think so.

Models are more than their output. Eric Topol, Yilan Wu, Alvin Liu, and Pearse Keane argue that the public health importance of healthcare AI now lies less in model performance than in platform integration depth. They use four models to make their case:

  • ChatGPT Health – 230M+ health queries a week but no direct care orchestration.
  • Amazon Health AI – extends further into orchestration by linking questions to primary care access (One Medical) and prescriptions (Amazon Pharmacy).
  • Claude for Healthcare – spans both ends by pairing a consumer arm that connects records with an enterprise arm touching prior auths, claims, and coding.
  • Ant Group’s Afu – the deepest integration, with booking, physician routing, pharmacy, and insurance all living inside Alipay (140M users).

Integrations run deeper every day. Once a system routes care and moves money, accuracy stops being the only question that matters.

  • Does the patient reach the right care? Does the prescription get filled? What do they pay out of pocket? Where do people drop out?

There’s also the structural risk. The authors give a few examples: 

  • Closed loops form when the same company interprets the symptom, routes the referral, fills the prescription, and collects the payment, so the checks that normally sit between those steps all live inside one P&L (Afu being the clearest case).
  • Linked records of eligibility, claims, and payments could become a profile of financial vulnerability that feeds targeting and pricing.
  • Triage power concentrates in a handful of interfaces that decide where people go first, with no hospital committee or human in the loop.

Do existing governance frameworks cover this? Not quite. Epic was the cautionary tale.

  • It became default infrastructure before governance caught up, hospitals adopted its models because they were already integrated, and its sepsis model ended up requiring clinicians to comb through 109 alerts for every true case.
  • The authors’ fix is to govern the pathways rather than the models themselves. Measure care completion, abandonment, out-of-pocket burdens, patient outcomes, and the effects that different social factors have on all of the above.

The Takeaway

When the same AI that’s recommending care is integrated with everything downstream, it might make sense to start governing pathways more than answers.

General-Purpose LLMs Outperform Healthcare-Specific Models

Posted on by Jason Barry

We might have just gotten our spiciest study of the year after new findings in Nature Medicine showed that general-purpose LLMs outperform specialized healthcare models straight out of the box.

It was a battle of the bots. Researchers pitted OpenEvidence and UpToDate Expert AI against three frontier models that anyone with a web browser can pull up in two seconds: GPT-5.2, Gemini 3.1 Pro, and Claude Opus 4.6.

The models were tested across three domains:

  • medical knowledge (MedQA)
  • expert clinician alignment (HealthBench)
  • 100 real physician queries (RCQ) scored by 12 blinded clinicians  

It was a clean sweep. The general-purpose LLMs outperformed the specialized models on all three evals, and by a healthy margin. This chart gets the point across.

  • On MedQA, Gemini led the pack with 97.4% accuracy (vs. 89.6% for OE and 88.4% for UTD). Fun fact, the frontier models had a huge advantage here since their training data included these exact questions (and answers).
  • On HealthBench, GPT-5.2 dominated with an 88%. It’s almost like OpenAI invented the benchmark.
  • The RCQs were probably the most clinically meaningful component, and all three frontier models took the podium here as well. It was a bit odd that the researchers didn’t share the specific questions, and OE definitely thought so too.

OpenEvidence hit back hard and fast. It went straight to its socials to let the world know that the study was not only poorly designed and biased, but that the authors had reached out for API access to help build a competing product. Request denied.

  • OE also pointing out the training data contamination issue with MedQA, and critiqued HealthBench for scoring responses based on subjective stylistic choices (in one example OE scored 20% “worse” because it didn’t use a specific email header).
  • The cherry on top was OE revealing that the real-world clinician queries were only added after peer reviewers flagged the study for having weak evidence. Big if true.

Obligatory disclaimer: the models were evaluated back in February, and the performance gap could easily be even wider today. 

The Takeaway

OpenEvidence and UpToDate didn’t become successful by being better AI developers than OpenAI and Anthropic. They did it by doing the things that don’t show up in benchmarks – curating sources of verifiable evidence, wrapping them in an interface that docs actually enjoy using, and earning their trust one question at a time. If anything, this study confirmed that those matter now more than ever.

AI Can Help Doctors Change Their Minds

Posted on by Jason Barry

A recent study out of Stanford explored whether doctors would revise their medical decisions in light of new AI-generated information, finding that docs are more than willing to change their minds despite being just as vulnerable to cognitive biases as the rest of us.

Here’s the set up published in Nature Communications Medicine: 

  • 50 physicians were randomized to watch a short video of either a white male or black female patient describing their chest pain with an identical script.
  • The physicians made triage, diagnosis, and treatment decisions using any non-AI resource.
  • The physicians were then given access to GPT-4 (which they were told was an AI system that had not yet been validated) and allowed to change their decisions.

The initial scores left plenty of room for improvement.

  • The docs achieved just 47% accuracy in the white male patient group.
  • The docs achieved a slightly better 63% accuracy in the black female patient group.

The physicians were surprisingly willing to change their minds based on the AI advice.

  • Accuracy scores with AI improved from 47% to 65% in the white male group.
  • Accuracy scores with AI improved from 63% to 80% in the black female group.

Not only were the physicians open to modifying their decisions with AI input, but doing so made them more accurate without introducing or exacerbating demographic biases.

  • Both groups showed nearly identical magnitudes of improvement (18%), suggesting that AI can augment physician decision-making while maintaining equitable care.
  • It’s worth noting that the docs used AI as more than a search engine, asking it to bring in new evidence, compare treatments, and even challenge their own beliefs [Table].

The Takeaway

Although having the doctors go first means that AI didn’t save them any time in this study – and actually increased time per patient – it showed that flipping the paradigm from “doctors checking AI’s work” to “AI helping doctors check their own work” has the potential to improve clinical decisions without amplifying biases.

AI Misses the Mark on Detecting Critical Conditions

Posted on by Jason Barry

Most health systems have already begun turning to AI to predict if patient health conditions will deteriorate, but a new study in Nature Communications Medicine suggests that current models aren’t cut out for the task. 

Virginia Tech researchers looked at several popular machine learning models cited in medical literature for predicting patient deterioration, then fed them datasets about the health of patients in ICUs or with cancer.

  • They then created test cases for the models to predict potential health issues and risk scores in the event that patient metrics were changed from the initial dataset.

AI missed the mark. For in-hospital mortality prediction, the models tested using the synthesized cases failed to recognize a staggering 66% of relevant patient injuries.

  • In some instances, the models failed to generate adequate mortality risk scores for every single test case.
  • That’s clearly not great news, especially considering that algorithms that can’t recognize critical patient conditions obviously can’t alert doctors when urgent action is needed.

The study authors point out that it’s extremely important for technology being used in patient care decisions to incorporate medical knowledge, and that “purely data-driven training alone is not sufficient.”

  • Not only did the study unearth “alarming deficiencies” in models being used for in-hospital mortality predictions, but it also turned up similar concerns with models predicting the prognosis of breast and lung cancer over five-year periods.
  • The authors conclude that a significant gap exists between raw data and the complexities of medical reality, so models trained solely on patient data are “grossly insufficient and have many dangerous blind spots.”

The Takeaway

The promise of AI remains just as immense as ever, but studies like this provide constant reminders that we need a diligent approach to adoption – not just for the technology itself but for the lives of the patients it touches. Ensuring that medical knowledge gets incorporated into clinical AI models also seems like a theme that we’re about to start hearing more often.

Study: AI is in the Eye of the Beholder

Posted on by Jason Barry

At a time when new healthcare AI solutions are getting unveiled every week, a study in Nature Machine Intelligence found that the way people are introduced to these models can have a major effect on their perceived effectiveness.

Researchers from MIT and ASU had 310 participants interact with a conversational AI mental health companion for 30 minutes before reviewing their experience and determining whether they would recommend it to a friend.

Participants were divided into three groups, which were each given a different priming statement about the AI’s motives:

  • No motives: A neutral view of the AI as a tool
  • Caring motives: A positive view where the AI cares about the user’s well-being
  • Manipulative motives: A negative view where the AI has malicious intentions

The results revealed that priming statements certainly influence user perceptions, and the majority of participants in all three groups reported experiences in line with expectations.

  • 88% of the “caring” group and 79% of the “no motive” group believed the AI was empathetic or neutral – despite the fact that they were engaging with identical agents.
  • Only 44% of the “manipulative” group agreed with the primer. As the authors put it, “If you tell someone to be suspicious of something, then they might just be more suspicious in general.”
  • As might be expected, participants who believed the model was caring also gave it higher effectiveness scores and were more likely to recommend it to a friend. That’s obviously relevant for those developing similar mental health chatbots, but a key insight for presenting any AI agent to new users.

An interesting feedback loop was also found between the priming and the conversation’s tone. People who believed the AI was caring tended to interact with it in a more positive way, making the agent’s responses drift positively over time. The opposite was true for those who believed it was manipulative. 

The Takeaway

The placebo effect is a well documented cornerstone of medical literature, but this might be the first study to bridge the phenomenon from sugar pill to AI chatbot. Although AI is often thought of as primarily an engineering problem, this research does a great job highlighting how human factors and the power of belief play a huge role in the perceived effectiveness of the technology.

Can Wearables Help Measure Patient Outcomes?

Posted on by Jason Barry

Researchers from the University of Edinburgh published a systematic review in Nature that aimed to determine the current evidence base and reporting quality for mobile digital health interventions (DHI) in the postoperative period following surgery.

Methodology – After screening 6,969 articles for patients undergoing surgeries where postoperative outcomes were measured using DHIs (defined as mobile technologies to improve health system efficiency and health outcomes), 44 studies were included in the final review.

Results – The review indicated that several types of mobile phone- or wearables-generated data can improve the assessment of postoperative recovery:

  • patient-reported outcome data (from validated self-report tools)
  • continuous activity data (from wearables)
  • combining remote assessment with active clinical prompts or patient advice

DHI Shortcomings – Studies included in the analysis demonstrated that DHIs may facilitate patient recovery following major operations and reduce inappropriate service use, although they also revealed issues with the current evidence base that should be addressed:

  • patients are rarely engaged in the development of DHIs
  • only one study was designed to engage patients in reviewing their own data
  • high levels of exclusion exist for patients without relevant mobile technology

Discussion

The increasing availability of high quality mobile technologies provides a new bridge between clinical services and patients’ homes, and while the authors of the study are optimistic about the technology, they stress the importance of improving reporting standards if its potential is to be fulfilled.

Going forward, the researchers suggest that studies of DHIs in postoperative settings seek to provide meaningful comparisons to non-DHI care in order to demonstrate clinical value, with particular attention paid to reporting quality so that equitable comparisons can be made to existing research.

Get the top digital health stories right in your inbox