|
Hippocratic, FDA Guidance, and AI Bias Blindness
January 13, 2025
|
|
|
|
|
Together with
|
|
|
|
“People talk about the workforce shortage of the future. There’s only a workforce shortage of the future if you don’t do anything today.”
|
|
Northwell CEO Michael Dowling
|
|
|
In the latest episode of the Digital Health Wire Show, Nabla’s Head of Information Security Brittney Harrell walks us through the checklist for choosing an ambient AI assistant with strong AI governance. Brittney shares a practical blueprint for safeguarding accountability and compliance in AI, as well as best practices on everything from model validation to cybersecurity.
|
|
|
|
A beautiful paper in Health Affairs brought us the first snapshot of AI oversight at U.S. hospitals, as well as a glimpse of the blindspots that are already adding up.
Data from 2,425 hospitals that participated in the 2023 AHA Annual Survey shed light on the differences in AI adoption and evaluation capacity at hospitals on both sides of a growing divide.
Two-thirds of hospitals reported using AI predictive models, a figure that’s likely only gone up over the last year. These models were most commonly used to:
- predict inpatient health trajectories (92%)
- identify high-risk outpatients (79%)
- facilitate scheduling (51%)
- perform a long tail of various administrative tasks
Bias blindness ran rampant. Although 61% of the AI-user hospitals evaluated accuracy using data from their own system (local evaluation), only 44% performed similar evaluations for bias.
- Those are some concerningly low percentages considering that models trained on external datasets might not be effective in different settings, and since AI bias is a surefire way to exacerbate health inequities.
- Hospitals that developed their own models, had high operating margins, and belonged to a health system were all more likely to conduct local evaluations.
There’s a digital divide between hospitals with the resources to build models tailored to their own patients and those who are getting these solutions “off the shelf,” which increases the risk that they were trained on data from patients that might look very different from their own.
- Only 54% of the AI hospitals designed their own models, while a larger share took the path of least resistance with algorithms supplied by their EHR developer (79%).
- Combine that with the fact that most hospitals aren’t conducting local evaluations of bias, and there’s a major lack of systematic protection preventing these models from underrepresenting certain patients or adding unfair barriers to care.
The authors conclude that policymakers should “ensure the use of accurate and unbiased AI for patients regardless of where they receive care… including interventions designed to connect underresourced hospitals to evaluative capacity.”
The Takeaway
Without the local evaluation of AI models, there’s a glaring blindspot in the oversight of algorithmic bias, and this study gives compelling evidence that more needs to be done to fill that void.
|
|
|
Bridging Care Gaps for Underserved Populations
Is your health system, rural health clinic, or federally qualified health center struggling to reach patients with obstacles to receiving in-person care? This Clear Arch Health whitepaper explores how combining RPM with VBC can help facilitate proactive interventions, address social determinants of health, and get the most out of new CMS reimbursement pathways.
|
|
Tailored Support, When Patients Need It Most
With BPM Pro 2’s Personalized Health Nudges, care teams can send tailored messages – such as positive reinforcement, medication reminders, or appointment alerts – precisely when patients are most receptive.
|
|
Reinforce Your Cybersecurity Defensive Line
Cybersecurity threats aren’t letting up, but an effective defense takes more than just firewalls and encryption. Check out Medallion’s recent Elevate session for expert insights into creating resilient systems and rebuilding trust after a breach.
|
|
- Hippocratic Series B: Hippocratic AI joined the unicorn club after locking in $141M of Series B financing at a $1.6B valuation. The capital will be used to expand Hippocratic’s AI agents for non-diagnostic, patient-facing applications to more verticals (pharma, payors) and new markets (EMEA, Southeast Asia, Latam). The announcement also included the launch of an AI Agent App Store where clinicians can collaborate with Hippocratic to design AI agents based on their own use cases and share in the revenue from future users.
- Neuroflow Acquired Intermountain Tech: Intermountain Health sold its home grown behavioral health analytics model to NeuroFlow after more than a decade of developing the predictive complexity algorithms. The model blends behavioral, medical, and social health data to equip providers with a comprehensive view of patient health risks and improve targeted care. NeuroFlow plans to incorporate the model into its existing analytics suite before rolling out new capabilities later this year.
- Utilization Management & Burnout: A study in the American Journal of Managed Care linked utilization management (UM) to significant barriers for physicians and their patients. Physicians reported that UM procedures for prior authorization (81%), step therapy (79%), and nonmedical switching (69%) negatively impacted their clinical and patient care. Over half (52%) reported spending 6 to 21+ hours per week on UM paperwork, 67% had experienced burnout, and 64% said UM contributed to that burnout.
- Abridge + Duke: Abridge opened the new year by inking an enterprise-wide agreement with Duke Health to deploy the ambient AI platform to 5,000 clinicians across 150+ primary and specialty clinics. The announcement lands just days after similar moves from other academic health systems, with UChicago Medicine and Johns Hopkins Medicine also recently signing on with Abridge.
- U.S. Healthcare Opinion Tumbles: A Gallup survey showed that Americans’ opinion of the healthcare industry just hit a 24-year low, with only 44% of adults viewing the overall state of care quality as good/excellent (only 28% said the same for U.S. health coverage). One of the most interesting takeaways was that 71% of adults consider the quality of their own healthcare to be good/excellent, and 65% were happy with their own coverage.
- Red Rover, Red Rover: EHR integration tech vendor Red Rover Health landed $4M in seed funding to help healthcare organizations streamline operations through real-time data exchange. The Red Rover Core platform’s open app store model leverages RESTful APIs to enable seamless integration for “best of breed” third-party apps when existing standards like HL7 and FHIR fail to meet bi-directional data flow needs.
- Hospital Operating Outlook Improves: For the first time in 27 months, Fitch lifted its credit outlook for the nonprofit hospital sector from “deteriorating” to “neutral.” Hospitals have seen “steady improvement” on operating margins due to providers seeing softer labor expense growth, stronger revenue, and better equity returns. Fitch predicts operating margins will continue improving to between 1% and 2% in 2025, though potential Medicaid cuts from the new administration could quickly revert the sector back to “deteriorating.”
- Dexcom’s Glucose AI: Dexcom launched “the first generative AI platform for glucose biosensing” to bring its Stelo CGM users personalized content based on their glucose levels, activity, and sleep. The AI feature leverages Google Cloud’s Vertex AI and Gemini models to analyze individual data, lifestyle, and glucose levels while providing metabolic health insights through the Stelo app. The upgrade also means that the app will now send text messages with the AI-driven advice.
- Off-Label Communication Guidelines: The FDA is pedal to the metal in 2025, following up its fresh AI draft guidance with new standards for medical firms on how to communicate “scientific information on unapproved uses of approved/cleared medical products” to healthcare providers. The guidance lays out how companies should convey off-label benefits of their products without running afoul of the agency’s marketing guardrails. The new standards update off-label communications guidelines first published in 2014.
- Dr. AI Will See You Now: In other regulatory news, a bill introduced to U.S. Congress would make it possible to classify AI and machine learning technologies as healthcare practitioners that are eligible to prescribe drugs. H.R. 238 would amend federal law to allow AI-based prescribing if “authorized by the State involved” and approved by the FDA, adding another wrinkle to the quickly-moving debate over medical AI regulation.
|
|
Join the Nabla Team!
Nabla is scaling up, and it’s looking for a Head of Marketing to lead its next phase of growth. This role will help broaden Nabla’s footprint and showcase the proven impact of ambient AI with a company dedicated to bringing joy back to the practice of medicine. Learn more and apply here.
|
|
Curate, Create, & Share at the Point of Care
It’s hard to find a more unique vantage point on AI than Playback Health co-founder Dr. Langer, whose role as the Chair of Neurosurgery at Lenox Hill allows him to actually use the platform he helped create. Head over to Dr. Langer’s latest blog to see how Playback is helping him spend more time caring for patients and enabling providers to “Curate, Create, & Share” at the point of care.
|
|
Top Systems Scale Primary Care With K Health
Leading health systems are turning to K Health’s AI-driven primary care solution to give their patients access to high-quality care with wait times measured in hours, not months. Find out why K Health is the only clinical AI company partnering with top systems to scale fully integrated primary care experiences.
|
|
|
|
|
