When the public uses AI chatbots for medical advice: what a randomised study found
Reviewed by Dr. Ram Kumar G, MBBS, MD (Internal Medicine), MPH (USA), on · Last updated
In short
Bean and colleagues randomised 1,298 adults to work through ten medical scenarios with one of three language models (GPT-4o, Llama 3 or Command R+) or with any source they would normally use. Tested alone, the models identified the relevant conditions in 94.9% of cases. People using the same models identified relevant conditions in under 34.5% of cases and chose the right course of action in under 44.2%, no better than the control group. Knowledge inside a model did not translate into better decisions by the people using it.
What the study did
| Feature | Detail |
|---|---|
| Design | Preregistered randomised study |
| Participants | 1,298 adults |
| Task | Ten medical scenarios: identify the likely conditions and choose a course of action (from self-care to calling an ambulance) |
| Groups | GPT-4o, Llama 3, Command R+, or a control group using any source they chose |
| Comparison | The same models answering the scenarios directly, without a person in the loop |
What it found
The authors traced much of the gap to the interaction: people often gave the models incomplete information, the models' answers mixed good and poor suggestions, and people did not reliably pick out the useful ones. The authors also report that standard benchmarks of medical knowledge, and simulated conversations in which another model played the patient, did not predict the failures seen with real people.
| Measure | Models alone | People using the models |
|---|---|---|
| Relevant conditions identified | 94.9% of cases | Under 34.5% |
| Correct course of action | 56.3% on average | Under 44.2% |
| Compared with the control group | Not applicable | No better |
Why it matters
- Exam-style benchmarks measure what a model knows, not what happens when a worried person types a few lines into it. The two can be very different.
- Simulated users are not a substitute for real ones. Evaluations of patient-facing tools need studies with the intended users.
- Choosing the right course of action is harder than naming a condition, and it is the decision that matters most for safety: whether to rest at home, see a doctor or go to the emergency department.
What the study does not show
- It tested general-purpose models, released before the study, in written scenarios. It does not show how purpose-built, regulated tools, newer models or clinician-facing systems perform.
- It does show that a model's knowledge alone is not evidence of benefit, which applies to every patient-facing AI tool.
Practical lessons
- For people using chatbots for health questions: treat them as a starting point, not a decision. Seek care for worrying or worsening symptoms, and call emergency services for emergencies.
- For clinicians: ask patients what they have read or been told by AI tools, and correct misunderstandings as you would for any online source.
- For developers and buyers: test with real intended users, measure decisions and outcomes, not just answer accuracy, and design the conversation to collect the information the model needs.
FAQ
Do AI chatbots help people make better medical decisions?
In this randomised study, no: people using three general-purpose language models did no better than people using their usual sources, even though the models alone knew the answers most of the time.
Which models were tested?
GPT-4o, Llama 3 and Command R+, compared with a control group free to use any source.
Why did people do worse than the models alone?
People often gave incomplete information, the models' responses mixed good and poor suggestions, and people did not reliably choose the right one.
Does this mean AI has no place in patient advice?
No, but it means patient-facing tools must be designed and tested with real users, with outcomes that matter, before they are relied on.
Where was the study published?
In Nature Medicine in 2026 (volume 32, pages 609–615), after a preprint in 2025.
Sources
Educational content only. This material teaches methods; it is not clinical advice. Code, models and examples here are not validated medical devices and must not be used to make decisions about real patients without proper clinical validation, governance and any required regulatory approval. Examples use synthetic or openly licensed de-identified data only. Never paste identifiable patient information into public AI tools. Mentions of models, tools or vendors are for identification, not endorsement.