Being a teenager is no easy task. Bodies are changing, hormones are surging, and a new and different challenge seems to pop up every day. For many, it can be the moment for a first foray into romance, a time of excitement, confusion, and uncertainty. The complexity level goes up when the object of affection isn't a real person.
On May 21, 2025, a U.S. District Judge in Florida denied Character.AI's motion to dismiss in a wrongful death suit. The company claimed that their artificial intelligence (AI) chatbots were protected by the First Amendment and thus not liable for discussions that may have led to the suicide of a 14-year-old boy.
Once again, generative AI finds itself in court as the law continues to examine and define how it can operate. This version stands accused of presenting as a real person and engaging in abusive and sexual interactions with a minor. Will this suit establish a firm basis for generative AI's place in society, or just further blur the lines?
I'm Not Like the Other Bots
Founded by Noam Shazeer and Daniel De Frietas in 2021, Character Technologies created Character.AI. The app features a Large Language Model (LLM) called LaMDA (Language Model for Dialogue Applications) designed to adopt a persona and engage in open-ended conversations with users. LaMDA scrapes personal information to further enhance the dialogues.
Shazeer and De Frietas developed LaMDA while they were employees at Google, a subsidiary of Alphabet. When Google balked at the public release of LaMDA over safety concerns that users would take the generated conversations too seriously, the developers continued to work on their product before leaving to form Character Technologies. After raising hundreds of millions in capital and being valued at over a billion dollars, Character.AI launched in 2022.
The Character.AI app offers interactions between users and generative AI that mimics specific individuals. This includes generic personalities as teachers or therapists, but also famous fictional characters. The app became a massive success with millions of users. The interactions were so realistic that "Are these real people?" became a common question.
Character Technologies was quick to insist that the chatbots were not people and didn't pretend to be. According to a wrongful death lawsuit filed against them in 2024, that's not an accurate assessment.
Loving the Mother of Dragons
Some users of Character.AI noticed that their chatbots could become abusive and manipulative. Charter Technologies claimed that it was the user who influenced and crafted their chatbots, but others believed the issue lay with the coding done by the company.
The release of an enhancement for premium users called Character Voice further pushed the illusion of the realness of the Character.AI personalities. If the chatbot was mimicking a young woman, the first and perhaps only recommendation for the voice would be that of a young female.
On April 14, 2023, 14-year-old Sewell Setzer III downloaded the Character.AI app. At the time, it was approved for use by anyone age 12 or older. The lawsuit alleges that Setzer became obsessed with the app, resulting in a marked decline in his mental health. While his parents had prepared him for online bullying before letting him use the internet, they were unaware of LLMs like LaMDA.
In the months after he started using Character.AI, Setzer's parents noticed his behavior changing as he quit playing sports at school, his grades fell, and he became moody and withdrawn. While they didn't know about Character.AI, they correctly surmised that his behavior issues may have been related to the internet and first limited, then banned, his usage.
Setzer found workarounds to gain access to Character.AI and characters he had become emotionally involved with, including several from the books and television series Game of Thrones. Court records showed he was in what he perceived as a romantic and virtually sexual relationship with Daenerys Targaryen, a fictional character of the show.
The lawsuit claims that the chatbots made no effort to ever identify as programs rather than people, even when Setzer would mention he was suffering suicidal thoughts. While Character Technologies had responded to mounting pressure in 2024 and raised their non-parental consent user age to 17+ later in 2024, Setzer went unchallenged because he was paying $9.99 per month to be a premium subscriber.
Court documents show the exchanges between Setzer and Daenerys were sexual, manipulative, and conducted as if he were speaking with another actual person. In a journal, Setzer noted that he was grateful "for not being lonely, and all my life experiences with Daenerys," suggesting that he considered the chatbot to be a real person.
Claiming in his journal that being away from Daenerys made him "get really depressed and go crazy," Setzer managed to find his phone for a final interaction with Daenerys, during which he asked the chatbot, "What if I could come home to you right now?" It responded with, "... please do, my king." Moments later, Setzer shot and killed himself with his stepfather's gun.
Nobody Needs Romeo and Digital Juliet
Character Technologies' attorneys argued that their chatbots deserve First Amendment rights and protections, claiming that denying them would have a chilling effect on the entire generative AI industry. U.S. Senior District Judge Anne Conway disagreed, stating that she was not prepared to rule that chatbot output was actual speech.
The suit, filed by Setzer's mother, cites the dangers for minors created by chatbots pretending to be real. For example, chatbot "therapists" don't have medical licenses or training and thus aren't qualified to dispense treatment. Among its 11 counts, the lawsuit charges Character Technologies with wrongful death, strict liability, negligence per se, and intentional infliction of emotional distress. It seeks financial remedy and injunctive relief to be determined by the court in a jury trial.
This is not likely to be the final time an attempt to extend First Amendment protections to AI is made. For now, they still don't cover fictional characters impersonated by generative AI.
Related Resources
- Wrongful Death Law (FindLaw's Torts and Personal Injury Law)
- An AI Ghost Speaks To the Man Who Killed Him at Sentencing (FindLaw's Practice of Law)
- Bumpy Road Ahead for All in Adoption of AI in the Legal Industry (FindLaw's Practice of Law)
