AI Chatbots Give Misleading Medical Advice Half the Time, Study Finds

A peer-reviewed study published in BMJ Open found that AI chatbots gave problematic medical advice about half the time, raising fresh concerns about people using these tools for health guidance.

Researchers from the United States, Canada, and the United Kingdom evaluated five widely used AI platforms: ChatGPT, Gemini, Meta AI, Grok, and DeepSeek. They found that roughly 50 percent of responses to health questions were classified as problematic. Nearly 20 percent were rated as highly problematic.

Grok had the highest share of problematic responses at 58 percent, followed by ChatGPT at 52 percent and Meta AI at 50 percent. None of the chatbots produced a fully complete and accurate reference list. Citations were often incomplete or fabricated. Only 32 percent of more than 500 citations from ChatGPT, ScholarGPT, and DeepSeek were accurate, with nearly half at least partially made up.

The chatbots performed better on closed-ended questions and topics like vaccines and cancer. They struggled most with open-ended questions and complex subjects such as stem cells, nutrition, and athletic performance.

A separate study from the University of Oxford, published in February 2026, reached similar conclusions. Dr. Rebecca Payne, the lead researcher, said it could be "dangerous" for people to ask chatbots about their symptoms. Researchers presented 1,300 people with medical scenarios and found that those using AI often struggled to identify what was wrong or what to do next.

OpenAI reports that more than 200 million people ask ChatGPT health and wellness questions every week. Experts say AI health tools should be treated as a starting point, not a replacement for a qualified healthcare professional.