Doctor-Driven AI, and AI-Driven Doctors

Just in time for one of healthcare’s biggest AI tradeshows, Doximity put out a survey that sharpened one of its biggest double-edged swords: doctor-driven AI leads to AI-driven doctors.

AI is everywhere. Doximity surveyed 3,379 physicians and found that 85% are now using AI for work at least weekly, up from 51% a year ago.

  • 52% of physicians use it daily, up from 37%.
  • Oncologists lead at 93% weekly, followed by Nephrology (65%) and Radiology (64%).
  • Even physicians in their 70s are at 66% weekly AI usage.

Trust runs deep. Physicians place the most trust in AI for literature search (94%), generating differentials (87%), and treatment recommendations (86%).

  • They’re most excited about AI’s potential in other areas: work-life balance (65%), a lighter admin workload (59%), and more time for patients (52%). The order says it all.

Skepticism also runs deep. More than half still name accuracy and reliability as a top concern, and three in four caught a factual inaccuracy in the past year.

  • Tech isn’t the only adoption barrier. A third of physicians cite legal and regulatory uncertainty as their top limiter, and 27% point to patient privacy or data security.

Tell me something I don’t know. Two-thirds of docs say AI has improved their skills and knowledge.

  • 81% say AI has taught them something clinically new (an updated guideline, an alternative diagnosis, an untried treatment), and 22% say it happens frequently.

What happens when AI is everywhere? Real-world skill could be nowhere, and 61% of docs worry about what AI will do to the skills of residents, fellows, and other trainees.

  • 30% cite skill erosion as a top limit on their own AI use, climbing to 43% among physicians 30 and under – the group using it most.

The Takeaway 

Doximity’s latest numbers show that AI is officially everywhere. That’s almost as spooky as the report’s cover image for residents worried about building their skills, but most of us would still rather have our doctor use a calculator if needed, even if the invention made them worse at mental math.

Google Goes After Hospitals With Gemini for Healthcare

Google just became the latest frontier AI player to lean in on the healthcare game.

Gemini for Healthcare is now available… to preview. The enterprise-grade solution extends the universal Gemini agent introduced last month into a HIPAA-ready worker that can deliver finished outcomes inside the tools payers and providers are already using.

  • Google calls it an “agentic layer” that runs full workflows across EHRs, billing platforms, clinical guidelines, and Medicare datasets without forcing teams to switch apps. 

Some cakes have four layers. This layer has four cakes:

  • Pre-configured skills for healthcare work developed alongside domain experts
  • Connectors to clinical, financial, and operational systems 
  • An open healthcare partner ecosystem to scale Gemini without lock-in
  • A governed platform for model selection that lets Gemini play nice with other models

That offers plenty to chew on. 

  • For providers, Gemini provides a patient’s history across care settings, charts are prepared before the visit, and authorization packets are assembled against payer policy before submission, so fewer claims come back denied.
  • For payers, Gemini checks requests against policies, organizes them for the reviewer, and explains benefits and costs to members from their own plan documents.

Google has some hungry friends. Launch partners include the biggest of the big – HCA, Highmark, Northwell, Optum Insight, Blue Cross Blue Shield of Massachusetts, and ASCO.

  • The connector list is also 30 names deep, including Epic and Oracle (read-only, no write-back), CMS (coverage rules), the NPI registry, and BMJ guidelines.

Big markets mean big competition. OpenAI opened the year with ChatGPT for Healthcare, rounded it out with an Epic integration and a public data plugin for PubMed and CMS coverage, then recently upgraded the workspaces with persistent agents.

  • Anthropic launched Claude for Healthcare at JPM with its own connectors to all of the same places, skills for prior auth and appeals, a partnership with OpenEvidence, and even named healthcare a growth market in its IPO filing.

The Takeaway

Google just became a massive contender in an already crowded arena. Every one of these platforms is now a place you can build a healthcare product on top of. The question now is what it will cost when it really matters?

Microsoft Maps the Road to Medical Superintelligence

Microsoft just wrapped up a huge week in medical AI. A new paper in Science showed MAI-DxO is more accurate than doctors, Microsoft AI charted its course to medical superintelligence, and the same team published a framework for what that actually looks like.

There’s a long road ahead. The Science paper formalizes SDBench, which turns 304 NEJM cases and 325 MIMIC-IV emergency admissions into step-by-step encounters where the diagnostician asks questions, orders priced tests, and decides when to commit. 

  • The setup is meant to emulate the day-to-day realities of how doctors operate, which means gradually gathering information by asking a whole lot of questions.

MAI-DxO plays a panel of virtual physicians that debate each case.

  • On complex NEJM cases, MAI-DxO scored 83% on diagnostic accuracy, versus 36% for physicians using references (and 22% without).
  • On ED cases, MAI-DxO notched 80%, again outshining the physicians at 47%.
  • The AI asked 25.5 questions before its first test. Physicians asked 5.6.

That might sound like medical superintelligence. Microsoft AI’s companion post frames the paper as a step toward that AI holy grail, but confirms there’s still a long road ahead. 

MAI CEO Mustafa Suleyman predicted medical superintelligence was two to three years away last summer, and is now taking steps to get us there even faster:

  • Microsoft is open-sourcing SDBench for select NEJM cases, with the Gatekeeper, Judge, and cost code included so others can test their models against it.
  • Microsoft is running a study at Beth Israel Deaconess Medical Center to have patients interact with MAI-DxO directly. That’s the same BIDMC working with Google’s AMIE team.
  • Microsoft is folding its health scientists and the Dragon Copilot and PowerScribe teams into Microsoft AI, alongside its Mayo Clinic frontier model collaboration.

Now the fine print, also written by Microsoft. A new BMJ Digital Health & AI paper defines medical superintelligence as surpassing the strongest human-led health systems across the full arc of care, then lists six dimensions that would entail: 

  • medical knowledge, diagnostic reasoning and triage, therapeutic decisions, longitudinal context, clinical coordination, and alignment with human values.

MAI-DxO only covers about two of the six. The other four involve sensing what patients actually want, treating them over years, and coordinating humans to make it happen

The Takeaway

Microsoft definitely gets credit for publishing its to-do list alongside its victory lap. Diagnosing NEJM cases is a nice box to check, but the remaining boxes are the ones without answer keys.

LLMs Recommend More Care Than Doctors 

Sometimes a serendipitous study in NEJM AI is just what the AI doctor ordered to round out a great debate with some empirical evidence.

AI alignment is a hot topic. Before the ink was done drying on our top story from last week, we were blessed with a peer-reviewed study comparing the cost of LLM-recommended care to actual care from treating physicians.

  • The headline stat: LLM-recommended care is about 3.2x as expensive as physician care.

Here’s how they found it. Researchers compared the diagnostic and referral plans of 24 leading LLMs (20 general-purpose, 4 medical-specific) with those actually made by physicians across 200 outpatient and ED visits. Every order was priced at 2026 Medicare rates.

  • AI care averaged $314 per visit, versus $97 for the physicians.
  • The gap was split almost evenly between extra tests ($105) and specialist referrals ($111).  

LLMs hate sitting still. While physicians ordered no diagnostics in two-thirds of encounters, AI felt the need to order something in every one of those cases – usually standard tests like metabolic panels, hemoglobin A1c, and lipids.

  • Further prompting helped but didn’t fix it. Telling the models to be “cost-aware” cut the gap to 2.5x, and a “parsimonious” prompt trimmed it to 1.4x.
  • The extra tests were judged guideline-concordant in about 20% of the cases where AI ordered more. The rest was just more.

Dr. Wachter called it. The UCSF Chair of Medicine had barely finished his essay on the AI alignment problem before this study arrived with “some empirical meat to put on its bones.”

  • He agrees with the study authors that the latest findings don’t argue against clinical AI, but rather that its value depends on how it’s deployed.
  • He also agrees that the next advance should be systems that recognize “what is enough while balancing patient safety with parsimony.”
  • The biggest difference lies in how we get there. Wachter believes “what is enough” isn’t something AI will discover on its own, it’s a value judgment that health systems, payers, regulators, and AI developers “will have to encode deliberately.”

The Takeaway

It probably isn’t great news for already-skyhigh spending that AI is recommending care that’s 3x more expensive than physicians. That said, practicing differently isn’t necessarily practicing worse, and this study left the question open whether AI is tilting toward wastefulness or thoroughness.

Healthcare’s Alignment Problem: A Preexisting Condition

As armies of AI agents march toward the healthcare industry, there’s a basic question that still needs answering. Who exactly are they fighting for? 

It’s an age-old question. This time around, UCSF Chair of Medicine Dr. Bob Wachter might have the answer.

  • Wachter’s new essay on Healthcare’s Alignment Problem starts with the definitions: “alignment” is the gap between human values and the values (or at least the behavior) of an AI system.
  • That’s obviously important in U.S. healthcare, where the principal-agent problem is especially acute. The person making a decision (the agent) doesn’t always share the goals of the person it’s affecting (the principal).
  • As AI decision support becomes more persuasive, Wachter makes the case that it will quietly encode one set of values and spread it at scale.

That’s the problem. AI agents are now saddled with the same challenge that human doctors face daily. Healthcare doesn’t have one “right” answer.

  • What’s best for hospital bottom lines might not be what’s best for patients, and early evidence shows that today’s AI is raising costs by helping providers optimize documentation. Wachter calls that a sideshow.

The most expensive tech in a hospital is the physician’s pen. Physician salaries are a small slice of spending, but decisions about what to test, prescribe, and treat drive $6T of spending every year. AI’s real cost is its influence on those decisions.

Wachter shares three examples:

  • A 45-year-old with a week of back pain. Guidelines say no MRI, but an AI tuned to avoid malpractice, or to a fee-for-service bottom line, orders it anyway. Overuse, now at scale.
  • A $250k Alzheimer’s drug that buys another month. An AI tuned to individual benefit recommends it every time. Can’t put a price on a life, but in this case it’s $250k/mo.
  • A 58-year-old with pre-diabetes whose doctor wants a GLP-1. An AI tuned to a payer steers her to lifestyle modification and a six-month recheck. It’s rationing that nobody in the room voted on.

What’s the fix? Transparency. Wachter believes AI should show the tradeoffs instead of a single answer, so stakeholders can make informed decisions rather than default ones.

  • Option A for the individual.
  • Option B for cost-effectiveness within the standard of care.
  • Option C for whatever else the moment calls for. 

The Takeaway

The costs versus outcomes debate comes down to values. AI agents are now pushing their values at scale, and Wachter makes a strong case that those values should be “visible, debated, and audited.”

Google’s AMIE Survives Peer Review, Mostly

The peer-reviewed version of Google’s study on AMIE’s real-world performance was just printed in The Lancet, and the headline safety result made it through intact. Some of the preprint’s bolder claims did not.

ICYMI. The setup is identical to the March preprint on the Articulate Medical Intelligence Explorer, Google’s “medical AI researcher” that’s better known as AMIE 

  • 114 urgent care patients at Beth Israel Deaconess Medical Center chatted with AMIE, which then created a list of potential diagnoses and management plans for PCPs to review prior to the appointments.
  • AMIE’s conversations were supervised live in case anyone needed to step in, and blinded evaluators graded the outputs afterward.

AMIE passed with flying colors. Key highlights:

  • Zero conversations needed a safety stop. Supervisors logged just one hallucination, where AMIE described a past surgery as upcoming.
  • AMIE’s differential included the final diagnosis in its top seven candidates for 90% of cases, and its top three for 75%.
  • Of the 44 PCPs who had AMIE’s outputs before their visits, 33 found it helpful for prep and 25 said it may have changed their approach.
  • Evaluators rated the conversations favorably in 87-100% of cases, and patient attitudes toward AI improved after chatting with AMIE.

Here’s what got cut. In March, Google claimed that AMIE’s differentials and management plans were the same quality as those produced by the PCPs – “without significant differences.”

  • The Lancet version drops that comparison entirely because of confounding: the clinicians had read AMIE’s transcript before writing their own assessments, which could have biased their diagnoses.
  • The management plan comparison survived, with no significant gap on appropriateness or safety, but PCPs taking the lead on practicality and cost-effectiveness.

That wasn’t the only change. Supervisor interventions grew from three in the preprint to five in the final paper, primarily because AMIE included a “malignancy” in three of its potential diagnoses.

  • One of those listed lymphoma, and the treating clinician believed “the patient might have had anxiety related to the output and assessed the interaction as somewhat harmful in hindsight.” The supervisor saw no visible distress and let it run.

The Takeaway

Zero safety stops in a real clinical setting is a nice victory for AMIE, and it will be even nicer if it can scale to more patients. Luckily, Google already has a nationwide study underway with Included Health to show it can. 

PHTI Says AI Is Automating the Wrong Things

The Peterson Health Technology Institute just published a roadmap for scaling high-value AI in healthcare, and the good news is that AI is currently doing exactly what it’s supposed to. Unfortunately, that’s also the bad news. 

Follow the incentives. PHTI sets the stage by explaining why existing fee-for-service models have led to AI solutions that fuel a broken system rather than improving it. Tough to argue with.

  • When health systems implement AI that optimizes coding for revenue, you’re going to see more Level 5 encounters. Not malicious, just tech doing its job. 
  • Payers are doing the same thing with prior authorization. AI squeezes every drop out of the rules that it’s given, which equates to more denials and lower cost per claim. 

PHTI says this is exactly what we asked for. When you incentivize volume and complexity, AI heads straight for it: prior auth, coding, claims, and appeals.

  • Focus follows revenue, and we’re currently on a direct course for a whole lot more of a rough status quo if incentives stay the same.

What’s the fix? Refocus reimbursement on outcomes over activity.

PHTI’s roadmap has four pillars to make that happen:

  • Understand what works – Define outcomes, compare AI solutions, and continuously monitor what actually improves outcomes. Results matter more than process, and they’re the quickest way to build trust.
  • Set rules for coverage and use – Clarify licensure, liability, oversight, and coverage for AI-enabled care. Get payers aligned so that proven tech has a defined path to payment.
  • Purchase effectively – Pay for outcomes rather than activity, with real-world performance reflected in vendor pricing power. If vendors are evaluated against transparent outcome measures, then those are exactly what’s likely to improve.
  • Redesign care delivery – Remove wasteful administrative processes instead of layering agents on top. PHTI envisions admin processes streamlining as a result of better care delivery, rather than setting its boundaries.

Incentivize outcomes. Not exactly a novel idea, but PHTI paints a scary picture of the AI-powered future where it doesn’t happen. AI is a multiplier, and now’s the time to make sure it’s multiplying the right variables.

The Takeaway

PHTI was right on cue with the AI horror stories, and it outlined exactly where the script needs to change if we want a happy ending.

AI Returns Arrived Faster and Higher Than Expected

Healthcare’s AI returns are already knocking on back office doors, and Bessemer’s new ROI Scorecard says it’s only a matter of time before they get to clinical workflows.

ROI arrived twice as fast as expected. According to the survey of 226 healthcare execs, AI returns are typically showing up within 12 months, averaging 3.5x and beating expectations in 40% of use cases.

  • That’s especially big news since most buyers underwrote 24-month paybacks, meaning that orgs still modeling that assumption are underinvesting against their own results.

The back office leads the pack. Revenue cycle management topped every measure, with 4x ROI and two-thirds of provider execs actively running semi- or fully autonomous agents.

Here’s the ROI breakdown by use case:

  • Provider revenue cycle: 4.0x
  • Payer claims: 3.4x
  • Payer provider network: 3.3x
  • Member engagement: 3.2x
  • Provider front office returns: 3.1x
  • Provider clinical: 2.9x

Automation leads straight to attrition. One of the most surprising stats was that half of orgs have already reduced headcount as a direct result of AI, or plan to within six months.

  • Reductions currently average 8% to 13% of the affected functions, which are a mirror image of the AI returns.
  • Among providers reporting cuts, 73% named revenue cycle and medical billing, which posted both the highest AI return and AI autonomy.

The three-year forecast is already here. Last year, 84% of execs expected AI to transform clinical treatment decisions within 3 to 5 years. A year later, 71% say it already has.

  • As always, trust is the biggest barrier for clinical AI. Nearly half of respondents (46%) trust AI to support clinical decision-making, yet 77% override the AI’s suggestions more than half the time. 

Most of the value still lies ahead. Execs expect “materially more” AI returns to be realized over the next 12 months, and the leading indicators can apparently already be seen in empty back office cubicles.

  • Over half of next year’s incremental AI investment will go toward deployment: 35% to scaling proven use cases and 20% to integrating them into existing workflows, versus 13% for infrastructure and 11% for foundation model licensing.

The Takeaway

While most of these surveys hint at where the AI puck is heading, BVP just showed where the return already is. Turns out it’s the exact same places that provider orgs are already cutting staff.

Integrating Consumer AI Into Healthcare Pathways

If AI recommends the right care and nobody is around to get it, did it really recommend the right care? A new perspective piece from Topol and friends in Nature Health doesn’t think so.

Models are more than their output. Eric Topol, Yilan Wu, Alvin Liu, and Pearse Keane argue that the public health importance of healthcare AI now lies less in model performance than in platform integration depth. They use four models to make their case:

  • ChatGPT Health – 230M+ health queries a week but no direct care orchestration.
  • Amazon Health AI – extends further into orchestration by linking questions to primary care access (One Medical) and prescriptions (Amazon Pharmacy).
  • Claude for Healthcare – spans both ends by pairing a consumer arm that connects records with an enterprise arm touching prior auths, claims, and coding.
  • Ant Group’s Afu – the deepest integration, with booking, physician routing, pharmacy, and insurance all living inside Alipay (140M users).

Integrations run deeper every day. Once a system routes care and moves money, accuracy stops being the only question that matters.

  • Does the patient reach the right care? Does the prescription get filled? What do they pay out of pocket? Where do people drop out?

There’s also the structural risk. The authors give a few examples: 

  • Closed loops form when the same company interprets the symptom, routes the referral, fills the prescription, and collects the payment, so the checks that normally sit between those steps all live inside one P&L (Afu being the clearest case).
  • Linked records of eligibility, claims, and payments could become a profile of financial vulnerability that feeds targeting and pricing.
  • Triage power concentrates in a handful of interfaces that decide where people go first, with no hospital committee or human in the loop.

Do existing governance frameworks cover this? Not quite. Epic was the cautionary tale.

  • It became default infrastructure before governance caught up, hospitals adopted its models because they were already integrated, and its sepsis model ended up requiring clinicians to comb through 109 alerts for every true case.
  • The authors’ fix is to govern the pathways rather than the models themselves. Measure care completion, abandonment, out-of-pocket burdens, patient outcomes, and the effects that different social factors have on all of the above.

The Takeaway

When the same AI that’s recommending care is integrated with everything downstream, it might make sense to start governing pathways more than answers.

McKinsey Does the Math on AI-Powered Care

The fine folks over at McKinsey are as bullish as ever on AI, with their latest report estimating that it can already handle over a fifth of U.S. outpatient care.

McKinsey sees AI reshaping healthcare across three vectors. This chart lays them out nicely.

  • Discursive care – reasoning and dialogue but not hands-on treatment (ex. intake, triage). 
  • Personalized pathways – AI trained on individual history to tailor treatment.
  • Autonomous interventions – self explanatory.

The report centers on the first vector. McKinsey analyzed 2024 commercial, Medicare, and Medicaid claims to flag what discursive AI could already own today:

  • 16-22% of all outpatient claims (13-19% of outpatient spending), ~3B claims/yr.
  • Most of that is evaluation visits with no or low-complexity care, 11-15% of claims.
  • The rest is interpretation of diagnostic and imaging results, 5-7%.

Big numbers. McKinsey expects discursive care to fuel the segments where demand goes unmet because of access barriers. Primary care is a prime example, with 92M people living in shortage areas and a 31 day average wait for an appointment.

  • AI stands to make PCPs more efficient decision-makers while improving access for patients, without needing to hire more docs to do it. It also means more diagnostics and specialist referrals.
  • That runs the risk of low-value care cascades, but McKinsey thinks AI would give care teams the confidence to break those cascades and manage more complex conditions in a primary care setting.

Mileage may vary. AI-powered discursive care threatens low-acuity visit volume for fee-for-service orgs, which would need follow-up care, referrals, and procedures to offset the revenue hit.

  • VBC orgs would fare better, using AI to improve their underlying care management economics by catching disease earlier, closing care gaps, and keeping patients engaged.

Who comes out ahead? Payers. With AI becoming the new front door to care, the top of the funnel is shifting from PCPs to whoever can feed the most patient data to the shiniest AI model.

  • McKinsey sees that as payers’ chance to go from financing care to orchestrating it, and says a payer-deployed AI clinician could even become the primary interface patients use to access care.
  • Definitely a hot take, and a potentially brutal hit for fax machine manufacturers. 

The Takeaway

It’s never too surprising when AI consultants say AI can do everything. Then again, that’s turning into a pretty popular opinion.

Get the top digital health stories right in your inbox

This content is exclusive to subscribers

Log in or join by entering your email below.

Completely free. Every Monday and Thursday.

DHW New Phone Image