As armies of AI agents march toward the healthcare industry, there’s a basic question that still needs answering. Who exactly are they fighting for?
It’s an age-old question. This time around, UCSF Chair of Medicine Dr. Bob Wachter might have the answer.
- Wachter’s new essay on Healthcare’s Alignment Problem starts with the definitions: “alignment” is the gap between human values and the values (or at least the behavior) of an AI system.
- That’s obviously important in U.S. healthcare, where the principal-agent problem is especially acute. The person making a decision (the agent) doesn’t always share the goals of the person it’s affecting (the principal).
- As AI decision support becomes more persuasive, Wachter makes the case that it will quietly encode one set of values and spread it at scale.
That’s the problem. AI agents are now saddled with the same challenge that human doctors face daily. Healthcare doesn’t have one “right” answer.
- What’s best for hospital bottom lines might not be what’s best for patients, and early evidence shows that today’s AI is raising costs by helping providers optimize documentation. Wachter calls that a sideshow.
The most expensive tech in a hospital is the physician’s pen. Physician salaries are a small slice of spending, but decisions about what to test, prescribe, and treat drive $6T of spending every year. AI’s real cost is its influence on those decisions.
Wachter shares three examples:
- A 45-year-old with a week of back pain. Guidelines say no MRI, but an AI tuned to avoid malpractice, or to a fee-for-service bottom line, orders it anyway. Overuse, now at scale.
- A $250k Alzheimer’s drug that buys another month. An AI tuned to individual benefit recommends it every time. Can’t put a price on a life, but in this case it’s $250k/mo.
- A 58-year-old with pre-diabetes whose doctor wants a GLP-1. An AI tuned to a payer steers her to lifestyle modification and a six-month recheck. It’s rationing that nobody in the room voted on.
What’s the fix? Transparency. Wachter believes AI should show the tradeoffs instead of a single answer, so stakeholders can make informed decisions rather than default ones.
- Option A for the individual.
- Option B for cost-effectiveness within the standard of care.
- Option C for whatever else the moment calls for.
The Takeaway
The costs versus outcomes debate comes down to values. AI agents are now pushing their values at scale, and Wachter makes a strong case that those values should be “visible, debated, and audited.”
