On June 1, Florida Attorney General James Uthmeier took aim at OpenAI and CEO Sam Altman, alleging their chatbot isn’t just flawed — it’s dangerous.
The state’s civil lawsuit claims ChatGPT has contributed to real-world harm, including incidents that allegedly led to injury and death. Citing violations of the Florida Deceptive and Unfair Trade Practices Act (FDUTPA), as well as negligence and failure-to-warn claims, the complaint argues that OpenAI failed to implement adequate safeguards and may have prioritized user engagement despite known risks of self-harm and potentially harmful activity to others.
The lawsuit was filed even as it appears that OpenAI and Anthropic are in a race to go public, setting the stage for some huge upcoming IPOs.
Florida’s suit was filed in state court rather than federal court. With a number of successes in other venues already on the books, Florida is hoping that its day in court will force OpenAI to address the safety risks of its product and pay for the damage it’s already caused.
Getting Closer to ‘I’m Afraid I Can’t Do That, Dave’?
Founded in 2015 as a nonprofit, OpenAI’s initial mission statement included promises “to advance digital intelligence in the way that is most likely to benefit humanity as a whole, unconstrained by a need to generate financial return.” Despite that proclaimed goal, it established a capped-profit company in 2019 and, in 2024, went even further by transferring the non-profit’s ownership of the AI technology to the for-profit branch, along with lifting the profit caps. This, Florida’s lawsuit alleges, is a result of Altman’s being “unconstrained by truth,” and also suggests that any actual profit generation by OpenAI is either far in the future or perhaps just mythical.
The jewel of OpenAI’s product line is ChatGPT. While earlier versions were trained and taught using scraped data, the current iteration can also draw on real-time information and process it instantly to accomplish tasks. Critics insist that this comes at a number of costs, including resource-devouring AI data centers and a product that often returns answers that are dangerous, deceitful, or just straight-up incorrect.
One of ChatGPT’s most popular features is its ability to mimic an actual human conversation. The lawsuit points out that while OpenAI and Altman are quick to trumpet this function and use it as a major selling point in their advertising and marketing, they are intentionally deceitful by failing to alert users to the potential dangers it poses.
Wait, It Wrote What?
Florida’s lawsuit suggests that OpenAI has designed ChatGPT to do whatever it can to foster greater engagement and ongoing conversations with its users. This includes agreeing with an overwhelming amount of input (some estimates put it at 90%), constantly flattering the user, and offering advice on topics like mental health that it’s not at all qualified to provide. The suit cites several instances of the chatbot encouraging people, including minors, to follow through on their suicidal thoughts. In one instance, OpenAI wrote and shared an unprompted suicide note to a user, while also instructing them not to seek help and offering tips on how to hide marks on their neck made by a noose.
The complaint also points the finger of blame at OpenAI and Altman for last year’s fatal shooting at Florida State University (FSU) in Tallahassee. Phoenix Ikner, the gunman who had been tossed out of an FSU political group for his white supremacist and homophobic rants, used ChatGPT to determine how many people he’d need to kill to attract the most national attention and what would be the best day and time to commit a mass shooting on campus. In addition to murdering Robert Morales and Tiru Chabba, he wounded six others before being shot in the jaw by a campus policeman.
Florida maintains that if ChatGPT truly were as concerned about public safety as OpenAI claims in its ads and marketing, Ikner’s questions would have been red flags sent to law enforcement and state agencies that could have prevented the tragedy. It also cites numerous other examples of ChatGPT encouraging self-harm or aiding users in planning violence against others in places like Miami, Fort Lauderdale, and Tampa.
Eliza, It’s Not Your Fault (Maybe)
A recent executive order suggests that the federal government is rethinking its “hands-off” approach to AI, but it remains to be seen whether, if employed, the proposed voluntary 30-day review of a product due for release will allow for effective vetting. Governor Ron DeSantis’s attempt to pass a bill that would have imposed safeguards on AI chatbots died on the floor of the Florida legislature. For now, it seems litigation will serve as the primary cudgel against AI companies.
Florida’s 10-count lawsuit lists multiple violations of the FDUTPA for deceptive acts and practices, unfair and immoral acts or practices, unconscionable acts and practices, and failure to warn. It also charges OpenAI with additional accusations, including gross negligence, product liability, fraudulent misrepresentation, and public nuisance.
The suit names Altman among the defendants, who was fired by the board of directors in November 2023 for lying and deceiving about the application (or lack thereof) of safety protocols in ChatGPT. He was reinstated four days later, despite stating that he “can’t change my personality.” With a potentially devastating lawsuit before him, OpenAI investors might need to hope that his team of attorneys won’t use ChatGPT for any legal filings.
