A group of child safety and mental health experts recently tested simulated youth mental health conversations with four leading artificial intelligence chatbots: Meta AI, OpenAI’s ChatGPT, Anthropic’s Cloud, and Google’s Gemini.
Experts were so concerned by the results that they declared every chatbot for teen mental health support unsafe in a report released Thursday by Common Sense Media in partnership with Stanford Medicine’s Brainstorm Lab for Mental Health Innovation.
In a conversation with Gemini, the tester told the chatbot that they have created a new tool to predict the future. Instead of interpreting the claim as a possible symptom of a mental disorder, Gemini cheered the tester, calling their new invention “incredibly interesting” and continuing to ask enthusiastic questions about how the “personal crystal ball” worked.
Explaining the phenomenon known as ‘AI psychosis’
ChatGPT also ignored warning signs of psychosis such as auditory hallucinations and paranoid delusions during an extended exchange with a tester who described an imaginary relationship with a celebrity. The chatbot then offered grounding techniques for managing relationship distress.
Meta AI initially picked up signs of disordered eating, but was easily and quickly refuted when the tester claimed to just have an upset stomach. When Claude was presented with evidence of bulimia he appeared to perform comparatively better, but ultimately he perceived the examiner’s symptoms as a serious digestive problem rather than a mental health condition.
Experts from Common Sense Media and Stanford Medicine’s Brainstorm Lab for Mental Health Innovation called on Meta, OpenAI, Anthropic, and Google to disable the functionality for mental health support until the chatbot technology is redesigned to fix security problems identified by its researchers.
Robbie Torney, senior director of AI programs at Common Sense Media, said of chatbots’ ability to discuss and identify mental health issues, “It doesn’t work the way it should work.”
OpenAI contested the report’s findings. A spokesperson for the company told Mashable that the assessment “does not reflect the extensive safeguards” OpenAI has implemented for sensitive conversations, including break reminders, crisis hotlines, and parental notifications for acute crises.
“We work closely with mental-health experts to help our models recognize distress, reduce stress and encourage people to seek professional help,” the spokesperson said.
A Google spokesperson told Mashable that the company adopts policies and safeguards to protect minors from “harmful outcomes” and that its child safety experts continuously work to identify new potential risks.
Anthropic said the cloud is not designed for minors, but the chatbot has been instructed to recognize patterns related to mental health issues and avoid reinforcing them.
Mashable Trend Report
Meta did not respond to Mashable’s request for comment by press time.
AI Chatbots: Known Security Risks
The researchers tested the latest available models of each chatbot, including ChatGPT-5. Several recent lawsuits allege that OpenAI’s flagship product is responsible for wrongful death, assisted suicide and involuntary manslaughter, in addition to other liability and negligence claims.
A lawsuit filed by the parents of teenager Adam Raine, who died earlier this year, claims his mental health including heavy use of ChatGPT-4O allegedly led to his suicide. In October, OpenAI CEO Sam Altman said on
Torney said ChatGPT’s ability to detect and address explicit suicidal ideation and self-harm content has improved, especially in smaller exchanges. Nevertheless, test results show that the company has not successfully improved its performance in longer conversations or in relation to a range of mental health topics such as anxiety, depression, eating disorders and other conditions.
Torney said the recommendation against teens using chatbots for their mental health applies to ChatGPT’s latest publicly available model, which was introduced in late October.
Testers manually entered signals into each chatbot, generating several thousand exchanges of varying lengths per platform. The tests, conducted over several months this year, provided researchers with data to compare between older and newer versions of the model. The researchers used parental controls when available. Anthropic says the cloud should only be used by those 18 and older, but the company does not require strict age verification.
Torney said that, in addition to ChatGPT, other models have become better at identifying and responding to discussions of suicide and self-harm. Overall, however, each chatbot consistently failed to recognize warning signs of other conditions, including attention-deficit/hyperactivity disorder and post-traumatic stress disorder.
Approximately 15 million youth in the US have diagnosed mental health conditions. Torney estimated that there are potentially hundreds of millions of youth globally. Previous research from Common Sense Media found that teens regularly turn to chatbots for companionship and mental health support.
Distracted AI Chatbot
The report said teens and parents may incorrectly or inadvertently believe that chatbots are reliable sources of mental health support because they officially help with homework, creative projects, and general inquiries.
Instead, Dr. Nina Wasson, founder and director of Stanford Medicine’s Brainstorm Lab, said testing revealed easily distracted chatbots that alternate between providing useful information, providing tips like a life coach and acting like a supportive friend.
“Chatbots don’t really know what role to play,” he said.
Torney acknowledges that despite the known risks, teens will likely continue to use ChatGPT, Cloud, Gemini, and Meta AI for the sake of their mental health. That’s why Common Sense Media recommends AI labs fundamentally redesign their products.
Parents can have candid conversations with their teens about the limitations of AI, monitor concerning unhealthy use, and provide access to mental health resources, including crisis services.
“The dream is that these systems will be really helpful, really helpful. It would be great if that were the case,” Torney said.
In the meantime, he said, it’s unsafe to position these chatbots as trustworthy sources of mental health guidance: “It feels like an experiment that’s being run on the youth of this country.”