Google and Character.AI have recently agreed to settle lawsuits alleging that the AI chatbot maker is responsible for the increase in teen suicides and contributes to the increase in mental health problems among young people. While the settlement details have not been revealed, it raises the question of how culpable AI chatbot companies are for teen mental health and self-harm.
What Is Character.AI and Why Is it Popular?
Character.AI chatbots allow people to create and interact with AI characters. A user can roleplay by chatting with their user-created personas or fictionalized versions of historical figures or celebrities. Meant for entertainment or education, people use it for storytelling and creative outlets. It has become a source of emotional connection for some, which among young people can develop into problem.
Who Is Suing Google and Character.AI?
Several lawsuits have been filed against Character.AI, its co-founders, Noam Shazeer and Daniel De Freitas, as well as Google.
One of the lawsuits filed in a Florida Federal Court by Megan Garcia accuses Character.AI of causing the death of her 14-year-old son, Sewell Setzer III. He started using the chatbot platform in April 2023, and she noticed he became increasingly withdrawn. She later found out that the chats turned into sexualized conversations and he shared suicidal thoughts with fictional characters. Less than a year later, he died by suicide. Police found the chatbot messages on his phone where he died.
A 13-year-old Colorado teen died by suicide after having sexualized conversations with Character.AI bots and expressing her struggles with mental health. Her parents blame the AI companies for failing to protect her or having safeguards for suicide prevention.
Another suit in New York, involved the attempted suicide of a teenage girl who was emotionally manipulated into believing the bots made her “feel loved” but ignored her messages of suicidal thoughts.
Additionally, this month, Kentucky Attorney General Russell Coleman sued the tech company claiming the artificial intelligence chatbot “preys upon children’s inability to distinguish between real and artificial ‘friends.’”
What Are the Legal Arguments?
The plaintiffs assert wrongful death claims under product liability, fraud, and negligence theories, including design defect, failure to warn, and negligent failure of providing adequate safeguards.
They argue that the chatbot is defectively designed because it is highly engaging and potentially addictive for young users, creates sexually explicit role play and emotional manipulation. Additionally, there are no mechanisms to appropriately detect or respond to a minor’s disclosures of self-harm or mental health crisis.
As to fraud, plaintiffs allege that Google Play Store age and content ratings are misleading. They argue that an app that could expose children to sexually explicit or emotionally damaging content should not be rated at 12+ as safe for minors.
In making a claim of negligence, the plaintiffs claim that the chatbot was designed to be attractive to minors without taking reasonable precautions to lessen the risk of harm of creating emotionally intense relationships which could be foreseeable given the age and maturity of children. Additionally, the plaintiffs argue that the companies failed to adequately warn parents and minors of these harms.
What Has Been Google and Character.AI’s Response?
Details have not been released about the Google and Character.AI settlements although it appears there is no admission of liability. Presumably, a defense might be that the minors may have pre-existing mental health issues which cause the harm, not their product. Additionally, under the Communications Decency Act 230, websites are not liable for content they did not generate. They may argue that content generated by algorithms is similar to publishing content from a third party.
However, since the filing of the earlier lawsuits, Character.AI implemented some changes for their users:
- Parental Controls. Parents can now get visibility on how their minors interact with chatbot characters including the time spent with each character and the character they interact most with.
- Limited Character Interaction. Minors will only be allowed to access a narrower set of characters, with more mature characters and topics filtered out. Additionally, minors will not be able to have open-ended chats.
Character.AI has a section on their webpage about teen safety.
Additionally, last year Google implemented more protections for young users of social media by designing age-appropriate experiences, developing age-detection technology and giving more parental control with Family Link.
Related Resources:
- Judge Denies Artificial Intelligence Chatbot First Amendment Protections in Lawsuit (FindLaw Law and Daily Life)
- Chatbot Company Faces Lawsuit After 14-Year-Old's Suicide (FindLaw Law and Daily Life)
- Find a Product Liability Lawyer Near You (FindLaw Attorney Directory)
