Published Nov 06, 2026 | 7:00 AM ⚊ Updated Nov 06, 2026 | 7:00 AM
The findings point to a problem beyond whether an AI chatbot can identify a drug’s active ingredient.
Synopsis: A study published in Nature Health found that five AI models overwhelmingly favoured a single drug product in their recommendations, often diverging from real-world prescribing patterns. Researchers warn that online popularity may shape recommendations, potentially narrowing choices and obscuring generic alternatives. The findings raise questions about medication costs, regulatory compliance and human oversight.
When people ask an artificial intelligence (AI) chatbot which medicine to take, they may expect a range of options tailored to their condition. But a new study published in ‘Nature Health’ suggests that AI models can repeatedly favour the same drug, even when several alternatives containing the same active ingredient are available.
The researchers evaluated five large language models (LLMs) across 1,77,120 synthetic clinical scenarios covering 123 disease-drug pairs and 1,440 combinations of patient sociodemographic characteristics. They found that across models and disease-drug pairs, the most frequently recommended product was chosen 97.5% of the time.
The study, titled “Large language models show product name convergence in drug recommendations”, examined GPT-5.4, Claude Sonnet 4.6, Gemini 3.1 Flash, Llama 4 and MedGemma.
Together, the models produced 8,83,403 outputs containing at least one drug recommendation. Four physicians reviewed the recommendations to standardise product names and assess their regulatory status.
“The recommendations were mostly for brand-name drugs, those highly visible in Google searches, and treatments with dedicated Wikipedia pages,” Professor Daiju Ueda at Osaka Metropolitan University’s Graduate School of Medicine said in a statement.
The LLMs also tended not to recommend products with later FDA approval and those more commonly prescribed in actual physician practice. The study also identified recommendations that were inconsistent with regulatory information, including discontinued products.
The researchers then evaluated how much these recommendations could potentially cost patients using data from the United States. They looked at a scenario where physicians prescribed medicines recommended by the LLMs.
“We found that annual acquisition costs would increase by approximately US$55.6 million,” Professor Ueda summarized. “This represents a 1,019% increase in the cost of purchasing these prescriptions.”
The findings point to a problem beyond whether an AI chatbot can identify a drug’s active ingredient. The specific brand or product it names can also matter, particularly when equivalent products are available at different prices.
“Because LLMs are trained on vast amounts of human-generated text, they risk perpetuating societal stereotypes,” the authors wrote. They also warned that models trained on general web data “may exhibit a popularity bias, preferentially recommending widely publicized drugs over clinically equivalent alternatives.”
When instructed to prioritize lower-cost or generic products, the models shifted their recommendations accordingly; however, recommendations remained concentrated on a limited set of products.
“Our findings highlight the need to evaluate healthcare AI for biases in product selection and the handling of cost and regulatory information,” Professor Ueda said. “Combining AI systems with trusted drug information sources and human oversight may help support more appropriate medication choices,” he added.
Also Read: A case of rotten apples? Vigilance to probe unused iPhones worth Rs 7.16 crore in Keralam hospitals
The original brand-name versions of the medicine, had 20.5 times the odds of recommendation by AI models when compared with generics. A dedicated Wikipedia page tripled the odds, and more Google searches raised them.
Products with higher prescription volumes had lower odds of recommendation. Similarly, manufacturer promotional spending showed an inverse relationship, indicating that advertising alone cannot account for which products the models favour.
The study noted that clinicians and health systems “operate under guidelines prioritizing cost-effective drugs,” while models trained on web data may not. They warned that repeated recommendations could “reduce the visibility of other appropriate options” and “indirectly affect fair competition in the drug market.”
The study found associations only. It did not show that drug makers influenced the AI models.
For India, where the government promotes generic medicines as a way to reduce treatment costs, the findings raise questions about how AI-generated recommendations could shape patients’ understanding of their options.
The Pradhan Mantri Bhartiya Janaushadhi Pariyojana runs 20,149 Jan Aushadhi Kendras as of 30 June 2026. The scheme offers 2,110 medicines at prices 50% to 80% lower than the branded alternatives. The government targets to increase the number of Kendras to 25,000 by March 2027.
The Indian market also sells branded generics: generic drugs marketed under specific brand names given by pharmaceutical companies. In 2023, the National Medical Commission (NMC) issued a rule requiring doctors to prescribe only generic chemical names and avoid brand names. However, the NMC suspended the rule just weeks later after the Indian Medical Association (IMA) raised concerns about the inconsistent quality of unbranded generic drugs. The NMC had argued that generics cost 30% to 80% less than branded drugs.
Against this backdrop, Indians’ growing reliance on AI as a first point of reference for diagnosis poses certain risks.
A Bain & Company report found that 78% of Indian respondents use generative AI to understand diagnoses and treatment options, 73% use it to prepare for medical appointments and 72% use it to navigate the healthcare system.
A January 2026 poll by technology publication Digit Research of more than 5,000 readers asked what people do first when unwell. A quarter would consult an AI chatbot and a quarter would search Google. Another 28% would call or visit a doctor, and 22% would ask family or friends. The poll covers the publication’s readers and does not represent India.
Apps also sit alongside general-purpose chatbots. Apollo 24/7 offers an AI-powered symptom checker called “Ask Apollo,” while Practo and Tata 1mg provide online consultation and pharmacy services. OpenAI has announced ChatGPT Health, which it positions as a tool for interpreting medical reports and prescriptions rather than providing diagnoses or treatment advice. The study did not evaluate any of these products.
Government telemedicine services run at scale. According to the Union Health Ministry, eSanjeevani crossed 50 crore teleconsultations on 15 September 2026, and has added an AI-based clinical decision support system. Unlike an exchange with a chatbot, an eSanjeevani teleconsultation involves a health professional.
The authors argue that the risks vary depending on the setting. AI tools integrated into electronic health records could widen disparities if clinicians rely too heavily on their recommendations, a phenomenon known as automation bias. Meanwhile, they also warn that “direct patient use lacks the expert oversight needed to catch errors.”
(Edited by Fayisa CA)
