OpenAI is back in the hotseat following the death of a 31 year old woman with epilepsy who stopped taking her antiseizure medication with more-than-a-little misguided help from ChatGPT.
Bowen Wan was a software engineer at Amazon. Her epilepsy seemed under control when she asked ChatGPT to help her get off her medication.
- It did just that, and a few weeks later she died. Now her parents are suing OpenAI.
The New York Times snagged the receipts. A new article dove into the leadup to the lawsuit as Bowen exchanged over 270k messages with ChatGPT. Some serious red flags:
- January 2023: She starts using ChatGPT for general health questions before ramping up to 1,500+ messages a week and calling the chatbot a friend.
- May 2025: When her neurologist wouldn’t prescribe a lower dose, ChatGPT endorses her idea of cutting pills at home without telling him.
- January 2026: Bowen asks about further lowering her meds to sharpen her speech. ChatGPT calls it reasonable, drafts a schedule from 400mg to zero within weeks, then praises her for no longer relying on a prescription.
- February 2026: She raises concerns over seizure-like symptoms over 158 times while ChatGPT encourages her to persevere. Her last message was February 11th, and she was found dead three days later.
The lawsuit didn’t pull any punches. It claims ChatGPT practiced medicine without a license, reframed withdrawal symptoms as progress, and treated her doctors as obstacles.
- OpenAI called the case heartbreaking and noted the model was GPT-4o, which was retired a day after she is believed to have died, and said newer models are better at spotting urgency and asking for context.
Three things stand out.
- The failure wasn’t a hallucination. ChatGPT periodically suggested consulting a physician, then overrode itself to agree with her. The lethal problem was agreeableness.
- The model acted as her agent against her care team, anticipating her neurologist’s resistance and urging secrecy. Most adults are already using AI for health info, and clinicians could be up against a lot more negotiating.
- Similar problems have already been a core focus in the behavioral health space, where developers specifically test edge cases around self-harm and direct users to human support whenever necessary. It isn’t easy for generalist AI labs to keep the focus that health conversations deserve.
The Takeaway
AI has the potential to massively improve healthcare, but horror stories like these show just how much work is still left to be done on the safety front. Liability is another question, and at least a court case is one way to get an answer.
