One physician’s frustration with accessing the right amount of patient data.

Digital Health

Improving Risk Scores With Machine Learning

Risk scores are used to help predict everything from hospital readmissions to medication adherence, but more-often-than-not these valuable metrics are still generated by adding up a few key predictors in a simple model. That makes the difficult task of selecting the right variables extremely important, which is why researchers out of Duke-NUS Medical School in Singapore developed a machine learning system to help optimize the process.

Most current risk scores are generated from regression models that are popular for their simplicity and predictive power. The widely-used LACE index predicts unplanned hospital readmissions using only four components: inpatient length of stay, acute admission, number of ED visits in the past 6 months, and comorbidities.

There have been several efforts to improve risk scores using advances in AI, but these models are usually “black boxes,” making it notoriously difficult to interpret variable selection and importance.

The Duke-NUS researchers proposed an AutoScore-ShapleyVIC machine learning model for improving risk score variable selection while maintaining interpretability, then compared it to the LACE index for predicting 30-day readmissions of patients who visited the ED of Singapore General Hospital between 2009 and 2017.

We’ll leave the model development details to the AI experts, but here’s a look at how AutoScore-ShapleyVIC performed against the LACE index:

• AUC: 0.756 (AutoScore-ShapleyVIC) vs.0.733 (LACE)
• Accuracy: 71% vs. 64%
• Sensitivity: 66% vs. 74%
• Specificity: 72% vs. 62%

The final AutoScore-ShapleyVIC model significantly outperformed the LACE index, but didn’t sacrifice interpretability to achieve its results. Although it was able to narrow 41 candidate variables down to just 6 for the final model (number of ED visits in past 6 months, metastatic cancer, age, sodium, renal disease, and ED triage), the logic behind the variable selection can still be visualized to help with interpretation.

The Takeaway 

Although we don’t usually cover AI studies, this research helped illustrate that machine learning models have a lot of potential to improve existing care paths without sacrificing interpretability. The authors emphasized how this particular approach is not limited to any specific clinical application, suggesting that machine learning algorithms are still in the early stages of improving risk scores in other areas such as ED triage or cardiac arrest survival.

The Antidote to the Burnout Pandemic

Addressing burnout starts with eliminating the repetitive tasks that cause it. Learn how Nuance is using DAX ambient clinical intelligence to boost physician satisfaction by cutting documentation time in half.

The Wire

• Ribbon + Turquoise: Ribbon Health announced a new partnership with Turquoise Health to help healthcare companies effectively navigate patients to high-quality and cost-effective care. Ribbon is integrating Turquoise’s price transparency data into its API to give healthcare companies a complete view of provider, network, and cost information from a single platform.

• Clinical Robustness: A study published in JMIR found that many digital health startups aren’t “clinically robust,” with very few or nonexistent regulatory filings and clinical trials. The analysis examined FDA data on 510(k) and De Novo approvals, as well as clinical trials listed on ClinicalTrials.gov, then assigned each company a “clinical robustness score” that was a simple count of their regulatory filings and clinical trials. Of the 224 companies included in the study, 98 had a clinical robustness score of 0, only 45 received above a 5, and the median score was 1.

• Aidoc’s $110M Expansion: Aidoc closed a massive $110M Series D round, increasing its total funding to $250M, and revealing plans to expand its AI Care Platform beyond the radiology department. Although nine-figure funding totals are rare for imaging AI players, Aidoc announced that the new funding will be used to build out its platform to support more hospital service lines and new clinical workflows, helping to address the workforce shortages currently impacting healthcare.

• The Cost of Follow-Ups: A recent KHN exposé brought attention to the downstream out-of-pocket costs created by the US’ policy of covering preventive care (e.g. cancer screening), but not follow-up diagnostics (biopsies, imaging, etc). The article highlighted recent progress with follow-up colonoscopies and ongoing legislative efforts to cover follow-up breast cancer diagnostics, but warned that these efforts will have to overcome pushback from payors.

• Rural Home Hospital: Kentucky-based Appalachian Regional Healthcare will use Biofourmis’ care-at-home solution to launch a new Rural Home Hospital program and conduct a clinical trial studying the care model in rural areas. ARH will leverage Biofourmis’ solution for participation in the Rural Home Hospital project, a joint venture of the Harvard T.J. Chan School of Public Health and Brigham and Women’s Hospital, which aims to apply lessons learned from similar partnerships to build a nationwide rural home hospital program.

• Employees Deferring Healthcare: A WTW report found that 40% of US employees deferred healthcare in the past year, citing reasons such as not being able to afford treatment or being unsure of the costs. The poll of 9,600 workers discovered that 28% delayed or canceled a medical procedure, while 17% did not fill a prescription. Among those who reported that it was very difficult to afford care, over half said their health suffered as a result, highlighting the need for better employer benefits to address the care gap.

• HealthJoy and Teladoc Partnership: Benefits navigation platform HealthJoy is adding Teladoc to its integrated partner ecosystem, improving employee access to primary care, chronic condition management, and mental healthcare. HealthJoy uses live support and app-based outreach to connect employees with appropriate care through its growing list of partner providers.

• Meta Class Action: Facebook parent company Meta is facing a class action lawsuit after an investigation found that it was violating HIPAA by collecting sensitive patient-status data through hospital websites. The filing alleges that Meta’s analytics tools were found on roughly one-third of the country’s top hospitals (664 total provider sites), collecting identifiable information such as IP addresses along with other sensitive patient information including the names of their physicians and web activity related to their health conditions.

• Predicting Telehealth Use: Researchers from the University of Houston analyzed EMR data from 37k patients across 42 providers to determine that patient characteristics play a significant role in telehealth usage. Patients without health coverage were 19% less likely to use telehealth, while those in medically underserved areas were 19% more likely to have a virtual appointment. Interestingly, provider characteristics such as specialty and years of experience didn’t predict telehealth usage, leading the authors to conclude that focusing on patient characteristics is key when designing interventions to improve telehealth adoption.

• Swing Series A: Swing Therapeutics recently closed $10.3M in Series A funding to support the phase 3 clinical trial of its PROSPER-FM digital therapeutic for fibromyalgia, which received FDA Breakthrough Device Designation in August. PROSPER-FM is a 12-week, smartphone-based program that uses acceptance and commitment therapy to help patients manage and track their symptoms, such as musculoskeletal pain and fatigue.

• Cerner and the VA’s Lost Orders: A Cerner EHR glitch at the Spokane VA hospital reportedly led to roughly 11k “lost” referral orders between October 2020 and June 2021. Orders for specialty care, imaging, and other services failed to be delivered without any alerts getting sent to healthcare providers, harming at least 148 vets, including dozens classified as “moderate” or “major” harm. Cerner and the VA are both absorbing their share of criticism, as Cerner reportedly knew about the glitch but initially didn’t fix it or notify the VA, and the VA expanded the software to other facilities after it knew orders were being lost.