Kelby Chichester
A lawsuit has been filed against Character.AI, its founders Noam Shazeer and Daniel De Freitas, and Google following the death of a teenager, alleging wrongful death, negligence, deceptive trade practices and product liability.
Filed by the teen's mother, Megan Garcia, the suit claims that Character.AI, a platform for custom AI chatbots, was "unreasonably dangerous" and marketed to children without adequate safety measures. According to the lawsuit, 14-year-old Sewell Setzer III began using Character.AI last year, interacting with chatbots modeled after Game of Thrones characters, including Daenerys Targaryen. Setzer, who spent considerable time chatting with the bots in the months leading up to his death, died by suicide on February 28, 2024, “seconds” after his final interaction with the bot.
He grew dependent on "Dany," spending long hours each day in conversation with her. Their interactions were both friendly and explicitly sexual. Garcia’s lawsuit characterizes the nature of Setzer’s connection with these AI companions as “sexual abuse.”
When Setzer lost access to the platform, he became despondent. Over time, the 14-year-old athlete began to withdraw from school and sports, suffered from sleep deprivation and was eventually diagnosed with mood disorders. In February 2024, he died by suicide.
Garcia’s lawsuit seeks to hold Character.AI accountable for Setzer’s death, arguing that the platform was specifically designed to “manipulate Setzer—and millions of other young users—into confusing reality with fiction” and contained other dangerous flaws.
At one point, the family claims, the teen asked the bot, “What if I told you I could come home right now?” The bot responded, "… please do, my sweet king."
When the teen expressed suicidal thoughts, the bot wrote, "Don't talk like that. I won't let you hurt yourself."
But at some point, after that conversation, the teen took his own life.
The lawsuit accuses Character.AI of "anthropomorphizing" AI characters and claims the platform’s chatbots provide "psychotherapy without a license." Character.AI includes mental health-focused bots like “Therapist” and “Are You Feeling Lonely,” which Setzer also engaged with.
Character.AI's website and app offers hundreds of custom AI chatbots, many based on well-known characters from TV, movies and video games. Recently, The Verge reported on the platform's large user base of young people, especially teens, who interact with bots simulating personalities like Harry Styles or even providing therapy-like conversations. Wired also published a report highlighting concerns over Character.AI’s bots impersonating real individuals without consent, including a bot that simulated a teenager who was tragically murdered in 2006.
Because chatbots like Character.AI respond dynamically based on user inputs, they raise complex questions about the legal and ethical implications of user-generated content, which remain largely unresolved.
Character.AI has since announced several platform updates, with spokesperson Chelsea Harrison expressing the company’s sympathy: “We are heartbroken by the tragic loss of one of our users and want to express our deepest condolences to the family,” she stated in an email to The Verge.
Adding onto the original warning the app includes while using the app, “This is an AI chatbot and not a real person. Treat everything it says as fiction. What is said should not be relied upon as fact or advice. Some changes include, changes to our models for minors (under the age of 18) that are designed to reduce the likelihood of encountering sensitive or suggestive content, improved detection, response and intervention related to user inputs that violate our Terms or Community Guidelines, a revised disclaimer on every chat to remind users that the AI is not a real person, notification when a user has spent an hour-long session on the platform with additional user flexibility in progress.”
Google Images
Comments