Google AMIE Shines in First Real-World Study

The gap between benchmark scores and real-world performance has been the theme of the year in AI research, so Google was right on cue with its first prospective clinical trial for AMIE using actual patients. 

Meet the Articulate Medical Intelligence Explorer. AMIE is Google’s flagship “medical AI researcher,” and it teamed up with Beth Israel Deaconess Medical Center to gauge performance in real clinical workflows.

  • 100 patients completed an AMIE interaction before their primary care visit, with AMIE taking medical histories and equipping patients with potential diagnoses to discuss with their PCP.
  • PCPs received the transcript, summary, and AMIE’s management plan prior to the visit. All interactions were monitored live by physicians trained to intervene if safety criteria weren’t met.

AMIE got a gold star. Not only were there zero safety stops across all 100 interactions, patients reported that their attitudes toward AI significantly improved after chatting with AMIE.

  • AMIE’s differential included the correct final diagnosis in 90% of cases (per chart review 8 weeks post-encounter), with 75% top-3 accuracy.
  • PCPs using AMIE reported increased visit preparedness in 75% of cases, as well as potential behavior change in nearly 60%.
  • The quality of AMIE’s differential diagnosis and management plan appropriateness was similar to PCPs, although PCPs won on management plan practicality and cost-effectiveness.

Other findings were less obvious. PCPs had the chart, the physical exam, and the pre-visit transcript, yet AMIE still matched them on differential quality and management safety without taking a single peak at the EHR.

  • That speaks to the ceiling (or lack there-of) for structured AI history-taking, and shows that AI is gearing up to improve patient care in more ways than just making predictions.
  • The fact that PCPs reported better visit preparedness and potential behavior change in over half of cases also highlights how AI can augment – not just replace – clinical reasoning.

The Takeaway

The distance between the bench and bedside is getting shorter, and Google’s AMIE results suggest that conversational AI in primary care is closer to reality than most people might think.

What is Google Doing in Health?

It was a big week for big tech in healthcare, with Google hosting its second annual The Check Up event to let teams across the company share updates on their latest health-related features.

The Search team had one of the biggest announcements of the event with the unveiling of a new feature that shows the appointment availability of nearby healthcare providers so users can easily schedule a visit. It’ll look something like this

  • Upon launching, the feature will first let users view open appointments at CVS MinuteClinics, but other providers can add their availability to their Google profiles through new scheduling integration with Kyruus and Stericycle.
  • Google emphasized that it won’t use any of the data for advertising purposes, a perennial concern as the search giant pushes further into healthcare.

YouTube is adding health source information panels designed to help viewers identify videos from authoritative sources, as well as health content shelves that more effectively highlight videos from these sources when searching for specific topics.

  • The features highlight Google’s plan to take an active role in combating health-related misinformation on its platforms, which has been a hot discussion topic since the start of the pandemic.

Fitbit announced that it submitted a request to the FDA for clearance to use photoplethysmography sensors to help detect atrial fibrillation after finding that the light-based PPG sensors identified the condition 98% of the time.

  • Fitbit already has the FDA’s go-ahead to use an ECG to monitor heart rhythms, but the PPG sensors are better suited for consistent long term monitoring.

Google’s Health AI team is teaming up with Northwestern Medicine to increase access to maternal health screening in low-income countries using a handheld ultrasound AI solution that helps lightly trained health workers and or even pregnant women conduct scans and interpret the results.

The Takeaway

Google’s latest feature updates illustrate the company’s revamped strategy of integrating health into its main service lines, which it announced following the dismantling of its Google Health business late last year. While none of the features are a huge shift individually, that isn’t necessarily a bad thing. Google seems committed to pivoting back to basics by bolstering the healthcare functions of Search and YouTube, which remain the foundation of its business.

Pandemic Lessons Led to Google Health Shakeup

The recent dismantling of Google Health following the departure of its chief, David Feinberg, MD, could easily have been interpreted as evidence that the company was retreating from healthcare.

To clear up any confusion, Google Chief Health Officer Karen DeSalvo, MD, spoke with Bloomberg about the search giant’s reorganization. She revealed that the changes reflect a shift in focus related to Google’s work during the pandemic, and that the company is in no way “retrenching on health.”

For Google, the pandemic was an unexpected crash course in health sector operations, expediting many of the lessons that could otherwise have taken years.

Dr. DeSalvo stated that the company’s work on services ranging from contact tracing to population mobility tracking played a large role in the decision to restructure its health unit.

  • Google’s old strategy revolved around consolidating the company’s wide ranging healthcare efforts, such as disease detection and clinical decision support, into a centralized product unit to be commercialized. Dr. Feinberg was hired in 2019 to lead the new division, Google Health, but his team members were disbanded into research and wearables units shortly after his departure.
  • Google’s new strategy is an effort to embed healthcare initiatives into its core products, such as Search and YouTube, rather than launching independent commercial services. This strategy is designed to have a wider influence on health by meeting consumers where they already are.

Industry Impact

With a majority of Google’s revenue coming from advertising, working with sensitive health data quickly attracts attention from regulators. One of Google Health’s early projects under Feinberg got particularly messy when a search tool created for the Ascension hospital network prompted a federal inquiry over data privacy concerns.

Although the Ascension search tool is still operational and secure, Dr. DeSalvo admits that the company must tread carefully when navigating the healthcare space, but believes that the reorganization will help to deliver superior medical care and human outcomes.

Get the top digital health stories right in your inbox