OpenAI in the Crosshairs of Wrongful Death Lawsuit 

OpenAI is back in the hotseat following the death of a 31 year old woman with epilepsy who stopped taking her antiseizure medication with more-than-a-little misguided help from ChatGPT.

Bowen Wan was a software engineer at Amazon. Her epilepsy seemed under control when she asked ChatGPT to help her get off her medication.

  • It did just that, and a few weeks later she died. Now her parents are suing OpenAI.

The New York Times snagged the receipts. A new article dove into the leadup to the lawsuit as Bowen exchanged over 270k messages with ChatGPT. Some serious red flags:

  • January 2023: She starts using ChatGPT for general health questions before ramping up to 1,500+ messages a week and calling the chatbot a friend.
  • May 2025: When her neurologist wouldn’t prescribe a lower dose, ChatGPT endorses her idea of cutting pills at home without telling him.
  • January 2026: Bowen asks about further lowering her meds to sharpen her speech. ChatGPT calls it reasonable, drafts a schedule from 400mg to zero within weeks, then praises her for no longer relying on a prescription.
  • February 2026: She raises concerns over seizure-like symptoms over 158 times while ChatGPT encourages her to persevere. Her last message was February 11th, and she was found dead three days later.

The lawsuit didn’t pull any punches. It claims ChatGPT practiced medicine without a license, reframed withdrawal symptoms as progress, and treated her doctors as obstacles.

  • OpenAI called the case heartbreaking and noted the model was GPT-4o, which was retired a day after she is believed to have died, and said newer models are better at spotting urgency and asking for context.

Three things stand out.

  • The failure wasn’t a hallucination. ChatGPT periodically suggested consulting a physician, then overrode itself to agree with her. The lethal problem was agreeableness.
  • The model acted as her agent against her care team, anticipating her neurologist’s resistance and urging secrecy. Most adults are already using AI for health info, and clinicians could be up against a lot more negotiating.
  • Similar problems have already been a core focus in the behavioral health space, where developers specifically test edge cases around self-harm and direct users to human support whenever necessary. It isn’t easy for generalist AI labs to keep the focus that health conversations deserve.

The Takeaway

AI has the potential to massively improve healthcare, but horror stories like these show just how much work is still left to be done on the safety front. Liability is another question, and at least a court case is one way to get an answer.

Microsoft Maps the Road to Medical Superintelligence

Microsoft just wrapped up a huge week in medical AI. A new paper in Science showed MAI-DxO is more accurate than doctors, Microsoft AI charted its course to medical superintelligence, and the same team published a framework for what that actually looks like.

There’s a long road ahead. The Science paper formalizes SDBench, which turns 304 NEJM cases and 325 MIMIC-IV emergency admissions into step-by-step encounters where the diagnostician asks questions, orders priced tests, and decides when to commit. 

  • The setup is meant to emulate the day-to-day realities of how doctors operate, which means gradually gathering information by asking a whole lot of questions.

MAI-DxO plays a panel of virtual physicians that debate each case.

  • On complex NEJM cases, MAI-DxO scored 83% on diagnostic accuracy, versus 36% for physicians using references (and 22% without).
  • On ED cases, MAI-DxO notched 80%, again outshining the physicians at 47%.
  • The AI asked 25.5 questions before its first test. Physicians asked 5.6.

That might sound like medical superintelligence. Microsoft AI’s companion post frames the paper as a step toward that AI holy grail, but confirms there’s still a long road ahead. 

MAI CEO Mustafa Suleyman predicted medical superintelligence was two to three years away last summer, and is now taking steps to get us there even faster:

  • Microsoft is open-sourcing SDBench for select NEJM cases, with the Gatekeeper, Judge, and cost code included so others can test their models against it.
  • Microsoft is running a study at Beth Israel Deaconess Medical Center to have patients interact with MAI-DxO directly. That’s the same BIDMC working with Google’s AMIE team.
  • Microsoft is folding its health scientists and the Dragon Copilot and PowerScribe teams into Microsoft AI, alongside its Mayo Clinic frontier model collaboration.

Now the fine print, also written by Microsoft. A new BMJ Digital Health & AI paper defines medical superintelligence as surpassing the strongest human-led health systems across the full arc of care, then lists six dimensions that would entail: 

  • medical knowledge, diagnostic reasoning and triage, therapeutic decisions, longitudinal context, clinical coordination, and alignment with human values.

MAI-DxO only covers about two of the six. The other four involve sensing what patients actually want, treating them over years, and coordinating humans to make it happen

The Takeaway

Microsoft definitely gets credit for publishing its to-do list alongside its victory lap. Diagnosing NEJM cases is a nice box to check, but the remaining boxes are the ones without answer keys.

LLMs Recommend More Care Than Doctors 

Sometimes a serendipitous study in NEJM AI is just what the AI doctor ordered to round out a great debate with some empirical evidence.

AI alignment is a hot topic. Before the ink was done drying on our top story from last week, we were blessed with a peer-reviewed study comparing the cost of LLM-recommended care to actual care from treating physicians.

  • The headline stat: LLM-recommended care is about 3.2x as expensive as physician care.

Here’s how they found it. Researchers compared the diagnostic and referral plans of 24 leading LLMs (20 general-purpose, 4 medical-specific) with those actually made by physicians across 200 outpatient and ED visits. Every order was priced at 2026 Medicare rates.

  • AI care averaged $314 per visit, versus $97 for the physicians.
  • The gap was split almost evenly between extra tests ($105) and specialist referrals ($111).  

LLMs hate sitting still. While physicians ordered no diagnostics in two-thirds of encounters, AI felt the need to order something in every one of those cases – usually standard tests like metabolic panels, hemoglobin A1c, and lipids.

  • Further prompting helped but didn’t fix it. Telling the models to be “cost-aware” cut the gap to 2.5x, and a “parsimonious” prompt trimmed it to 1.4x.
  • The extra tests were judged guideline-concordant in about 20% of the cases where AI ordered more. The rest was just more.

Dr. Wachter called it. The UCSF Chair of Medicine had barely finished his essay on the AI alignment problem before this study arrived with “some empirical meat to put on its bones.”

  • He agrees with the study authors that the latest findings don’t argue against clinical AI, but rather that its value depends on how it’s deployed.
  • He also agrees that the next advance should be systems that recognize “what is enough while balancing patient safety with parsimony.”
  • The biggest difference lies in how we get there. Wachter believes “what is enough” isn’t something AI will discover on its own, it’s a value judgment that health systems, payers, regulators, and AI developers “will have to encode deliberately.”

The Takeaway

It probably isn’t great news for already-skyhigh spending that AI is recommending care that’s 3x more expensive than physicians. That said, practicing differently isn’t necessarily practicing worse, and this study left the question open whether AI is tilting toward wastefulness or thoroughness.

Healthcare’s Alignment Problem: A Preexisting Condition

As armies of AI agents march toward the healthcare industry, there’s a basic question that still needs answering. Who exactly are they fighting for? 

It’s an age-old question. This time around, UCSF Chair of Medicine Dr. Bob Wachter might have the answer.

  • Wachter’s new essay on Healthcare’s Alignment Problem starts with the definitions: “alignment” is the gap between human values and the values (or at least the behavior) of an AI system.
  • That’s obviously important in U.S. healthcare, where the principal-agent problem is especially acute. The person making a decision (the agent) doesn’t always share the goals of the person it’s affecting (the principal).
  • As AI decision support becomes more persuasive, Wachter makes the case that it will quietly encode one set of values and spread it at scale.

That’s the problem. AI agents are now saddled with the same challenge that human doctors face daily. Healthcare doesn’t have one “right” answer.

  • What’s best for hospital bottom lines might not be what’s best for patients, and early evidence shows that today’s AI is raising costs by helping providers optimize documentation. Wachter calls that a sideshow.

The most expensive tech in a hospital is the physician’s pen. Physician salaries are a small slice of spending, but decisions about what to test, prescribe, and treat drive $6T of spending every year. AI’s real cost is its influence on those decisions.

Wachter shares three examples:

  • A 45-year-old with a week of back pain. Guidelines say no MRI, but an AI tuned to avoid malpractice, or to a fee-for-service bottom line, orders it anyway. Overuse, now at scale.
  • A $250k Alzheimer’s drug that buys another month. An AI tuned to individual benefit recommends it every time. Can’t put a price on a life, but in this case it’s $250k/mo.
  • A 58-year-old with pre-diabetes whose doctor wants a GLP-1. An AI tuned to a payer steers her to lifestyle modification and a six-month recheck. It’s rationing that nobody in the room voted on.

What’s the fix? Transparency. Wachter believes AI should show the tradeoffs instead of a single answer, so stakeholders can make informed decisions rather than default ones.

  • Option A for the individual.
  • Option B for cost-effectiveness within the standard of care.
  • Option C for whatever else the moment calls for. 

The Takeaway

The costs versus outcomes debate comes down to values. AI agents are now pushing their values at scale, and Wachter makes a strong case that those values should be “visible, debated, and audited.”

2026 State of Digital Health Purchasing

Peterson Health Technology Institute’s 2026 State of Digital Health Purchasing offered insider access to the minds of executives that are actively spending on digital health solutions.

PHTI surveyed 321 people with buying power across health plans, employers, and health systems to shine light on everything from purchasing approaches to contracting strategies.

Respondents revealed that AI use is wide. That doesn’t mean deployment is deep. 

  • Health systems and health plans have both homed in on solutions for clinical documentation, with 93% and 86% of decisionmakers reporting some use.
  • That said, only 41% of systems and 26% of plans reported enterprise-level deployment of the technology.

Same budgets, different targets. A third of purchasers said their digital health budgets have been consistent over the past two years, but they’ve started honing those budgets on solutions with better returns.

  • Patient outcomes surprisingly ranked as the top priority. 97% of purchasers prioritize technology that improves access and clinical performance.
  • It was nice to see patients edge out revenue, even by a slim margin. 96% of respondents looked for solutions that improve the financial performance of their org.
  • That was especially true within crowded markets surrounding the top three care priorities: diabetes, primary care, and mental health.

The focus is coming by necessity. Many vendors apparently aren’t meeting expectations. 

  • Cost-savings expectations weren’t met for 31% of health plans, 28% of employers, and 22% of health systems.
  • Engagement has also become increasingly important, yet 47% of purchasers report enrolling <25% of eligible members. 
  • Poor engagement and costs also ranked in each purchasing group’s top three reasons for switching vendors — and the main reasons they’d be willing to spend more.

Those numbers are affecting contracts. Purchasers are incentivizing vendors to hold up their end of the bargain by hanging their payments on outcomes.

  • More than two-thirds of employers use performance-based contracts, and 85% of contracts hold a quarter (or more) of fees contingent to performance.
  • Vendors also have less time to prove their worth, as 70% of contracts only last one to two years.

The Takeaway

Purse strings aren’t getting any looser, and more vendors are competing for the same purchers. PHTI expects capital to start flowing toward those with numbers that back their worth… quickly.

Google’s AMIE Survives Peer Review, Mostly

The peer-reviewed version of Google’s study on AMIE’s real-world performance was just printed in The Lancet, and the headline safety result made it through intact. Some of the preprint’s bolder claims did not.

ICYMI. The setup is identical to the March preprint on the Articulate Medical Intelligence Explorer, Google’s “medical AI researcher” that’s better known as AMIE 

  • 114 urgent care patients at Beth Israel Deaconess Medical Center chatted with AMIE, which then created a list of potential diagnoses and management plans for PCPs to review prior to the appointments.
  • AMIE’s conversations were supervised live in case anyone needed to step in, and blinded evaluators graded the outputs afterward.

AMIE passed with flying colors. Key highlights:

  • Zero conversations needed a safety stop. Supervisors logged just one hallucination, where AMIE described a past surgery as upcoming.
  • AMIE’s differential included the final diagnosis in its top seven candidates for 90% of cases, and its top three for 75%.
  • Of the 44 PCPs who had AMIE’s outputs before their visits, 33 found it helpful for prep and 25 said it may have changed their approach.
  • Evaluators rated the conversations favorably in 87-100% of cases, and patient attitudes toward AI improved after chatting with AMIE.

Here’s what got cut. In March, Google claimed that AMIE’s differentials and management plans were the same quality as those produced by the PCPs – “without significant differences.”

  • The Lancet version drops that comparison entirely because of confounding: the clinicians had read AMIE’s transcript before writing their own assessments, which could have biased their diagnoses.
  • The management plan comparison survived, with no significant gap on appropriateness or safety, but PCPs taking the lead on practicality and cost-effectiveness.

That wasn’t the only change. Supervisor interventions grew from three in the preprint to five in the final paper, primarily because AMIE included a “malignancy” in three of its potential diagnoses.

  • One of those listed lymphoma, and the treating clinician believed “the patient might have had anxiety related to the output and assessed the interaction as somewhat harmful in hindsight.” The supervisor saw no visible distress and let it run.

The Takeaway

Zero safety stops in a real clinical setting is a nice victory for AMIE, and it will be even nicer if it can scale to more patients. Luckily, Google already has a nationwide study underway with Included Health to show it can. 

PHTI Says AI Is Automating the Wrong Things

The Peterson Health Technology Institute just published a roadmap for scaling high-value AI in healthcare, and the good news is that AI is currently doing exactly what it’s supposed to. Unfortunately, that’s also the bad news. 

Follow the incentives. PHTI sets the stage by explaining why existing fee-for-service models have led to AI solutions that fuel a broken system rather than improving it. Tough to argue with.

  • When health systems implement AI that optimizes coding for revenue, you’re going to see more Level 5 encounters. Not malicious, just tech doing its job. 
  • Payers are doing the same thing with prior authorization. AI squeezes every drop out of the rules that it’s given, which equates to more denials and lower cost per claim. 

PHTI says this is exactly what we asked for. When you incentivize volume and complexity, AI heads straight for it: prior auth, coding, claims, and appeals.

  • Focus follows revenue, and we’re currently on a direct course for a whole lot more of a rough status quo if incentives stay the same.

What’s the fix? Refocus reimbursement on outcomes over activity.

PHTI’s roadmap has four pillars to make that happen:

  • Understand what works – Define outcomes, compare AI solutions, and continuously monitor what actually improves outcomes. Results matter more than process, and they’re the quickest way to build trust.
  • Set rules for coverage and use – Clarify licensure, liability, oversight, and coverage for AI-enabled care. Get payers aligned so that proven tech has a defined path to payment.
  • Purchase effectively – Pay for outcomes rather than activity, with real-world performance reflected in vendor pricing power. If vendors are evaluated against transparent outcome measures, then those are exactly what’s likely to improve.
  • Redesign care delivery – Remove wasteful administrative processes instead of layering agents on top. PHTI envisions admin processes streamlining as a result of better care delivery, rather than setting its boundaries.

Incentivize outcomes. Not exactly a novel idea, but PHTI paints a scary picture of the AI-powered future where it doesn’t happen. AI is a multiplier, and now’s the time to make sure it’s multiplying the right variables.

The Takeaway

PHTI was right on cue with the AI horror stories, and it outlined exactly where the script needs to change if we want a happy ending.

AI Returns Arrived Faster and Higher Than Expected

Healthcare’s AI returns are already knocking on back office doors, and Bessemer’s new ROI Scorecard says it’s only a matter of time before they get to clinical workflows.

ROI arrived twice as fast as expected. According to the survey of 226 healthcare execs, AI returns are typically showing up within 12 months, averaging 3.5x and beating expectations in 40% of use cases.

  • That’s especially big news since most buyers underwrote 24-month paybacks, meaning that orgs still modeling that assumption are underinvesting against their own results.

The back office leads the pack. Revenue cycle management topped every measure, with 4x ROI and two-thirds of provider execs actively running semi- or fully autonomous agents.

Here’s the ROI breakdown by use case:

  • Provider revenue cycle: 4.0x
  • Payer claims: 3.4x
  • Payer provider network: 3.3x
  • Member engagement: 3.2x
  • Provider front office returns: 3.1x
  • Provider clinical: 2.9x

Automation leads straight to attrition. One of the most surprising stats was that half of orgs have already reduced headcount as a direct result of AI, or plan to within six months.

  • Reductions currently average 8% to 13% of the affected functions, which are a mirror image of the AI returns.
  • Among providers reporting cuts, 73% named revenue cycle and medical billing, which posted both the highest AI return and AI autonomy.

The three-year forecast is already here. Last year, 84% of execs expected AI to transform clinical treatment decisions within 3 to 5 years. A year later, 71% say it already has.

  • As always, trust is the biggest barrier for clinical AI. Nearly half of respondents (46%) trust AI to support clinical decision-making, yet 77% override the AI’s suggestions more than half the time. 

Most of the value still lies ahead. Execs expect “materially more” AI returns to be realized over the next 12 months, and the leading indicators can apparently already be seen in empty back office cubicles.

  • Over half of next year’s incremental AI investment will go toward deployment: 35% to scaling proven use cases and 20% to integrating them into existing workflows, versus 13% for infrastructure and 11% for foundation model licensing.

The Takeaway

While most of these surveys hint at where the AI puck is heading, BVP just showed where the return already is. Turns out it’s the exact same places that provider orgs are already cutting staff.

General Medicine Lands $120M to Build the Store

Everybody is using AI to see what they need to improve their health, but that doesn’t mean they have a store that’s magically stocked with that care. That is unless General Medicine builds it with the $120M Series B it just raised.

CEO TJ Parker has been here before. When he first noticed healthcare’s lack of great consumer experiences circa 2012, he decided to build his own – PillPack.

  • PillPack didn’t grow to hundreds of millions in revenue through better contracts or technology, it got there by creating a pharmacy that consumers “actually loved using.”
  • What do consumers love? Seeing actual prices, comparing their options, and choosing the one that works best for them. 

General Medicine brings that same ethos to… general medicine. Who would’ve thought.

Say Dr. AI tells you to check your cholesterol. To turn that advice into care, you still have to identify the right test, determine if you need an order, and find somewhere you can go to get it. 

  • If the result comes back high, the same problem repeats with treatment.
  • What you need is a store. Easier said than done in healthcare (per usual).

Take the cholesterol example:

  • Clinicians, payers, and labs might all have different definitions for the same test.
  • Two tests that look identical can be clinically different.
  • Other times they’re identical in every way that matters except billing. 

General Medicine is the missing product catalog. It put 2,900 distinct medications, labs, telehealth visits, and select in-person care in a single searchable store.

  • GM doesn’t provide all that care itself. It lets you browse options from its own medical group, or connect with clinicians anywhere in the country through the same storefront.
  • That works because GM defines the product first, then maps everything back to it. The store then combines benefits, contracted rates, and manufacturer discounts to show customers what their care will actually cost before they decide.

Product definitions are only half the problem. You also have to understand your customer. GM gathers an average of 11 years of health history for each of its customers.

  • When you search for a cholesterol test, GM already has your history and coverage information, so you can easily compare labs and treatment options if needed.
  • For $15, a GM clinician will review your history and results, determine which treatments are appropriate, and help make any changes if there’s side effects down the road.

Sound too good to be true? Maybe, but a16z believed in the vision enough to lead a nine-figure round, and the store won’t have any problem getting people through the door if it can recreate some of the consumer magic we’ve already seen with certain packaged pill companies.

The Takeaway

Healthcare has always been organized by who delivers care, not what someone is trying to accomplish. General Medicine is building its store the other way around, and $120M should be enough to tell which way consumers prefer.

Is Supersized Healthcare Spending Peaking Premiums?

Nearly 12% of working-age U.S. adults are now without insurance coverage as rising premiums empty wallets. While many point the finger at insurers, a new JAMA Health Forum study revealed the culprit may be health spending on the whole.

  • As of 2024, the median household income was $83.7k, with annual premiums for employer-sponsored individual and family plans at $8.9k and $25.6k, respectively.
  • In the same year, the number of uninsured Americans under 65 years old rose to nearly 27M, reversing an almost five-year trend of declines.
  • More than 60% said they were uninsured because costs were too high.

Now, researchers from Yale and the University of Wisconsin-Madison compiled data on premiums, markups, and spending for non-self-insured employers to trace what went wrong, finding that between 2011 and 2024:

  • Average insurance premiums grew by 78.4% ($3.1k).
  • Average health spending grew by 84.2% ($2.8k).
  • The surprise was that insurer markups dropped from 18.6% in 2011 to 14.9% in 2024. 

Health spending might’ve been passing the buck – rather than insurers. Skyhigh healthcare costs accounted for 91% of the overall growth in premium averages.

  • These numbers come on the heels of a survey which revealed nearly half of Americans were suspicious of insurers steering ballooning health spending.
  • Still, the researchers noted that, since premiums are written to cover anticipated costs or losses, they expected them to be responsive to changes in healthcare costs. 

So, whodunnit? That’s a bit harder to pin, but other research shows it doesn’t seem like things are slowing down soon. 

  • The incidence of complex conditions is increasing as baby boomers age, and Medicare spending is increasing alongside it.
  • Staffing shortages force fewer workers to provide more care, with hospitals increasing wages to try to incentivize overworked clinicians to stay.
  • Those aren’t the only factors, but they’re certainly connected. The demand for care is increasing as the supply of workers is decreasing – and that doesn’t bode well for spending as a whole.

The Takeaway

Supercharged healthcare spending – not insurer markups – seems to be accounting for most of the increases in insurance premiums, and things may get worse before they get better.

Get the top digital health stories right in your inbox