Character.AI and Google sued after chatbot-obsessed teen’s death

A suit has been filed against Character.AI, its founders Noam Shazeer and Daniel De Freitas, and Google successful the aftermath of a teenager’s death, alleging wrongful death, negligence, deceptive commercialized practices, and merchandise liability. Filed by the teen’s mother, Megan Garcia, it claims the level for customized AI chatbots was “unreasonably dangerous” and lacked information guardrails portion being marketed to children.

As outlined successful the lawsuit, 14-year-old Sewell Setzer III began utilizing Character.AI past year, interacting with chatbots modeled aft characters from The Game of Thrones, including Daenerys Targaryen. Setzer, who chatted with the bots continuously successful the months earlier his death, died by termination connected February 28th, 2024, “seconds” aft his past enactment with the bot.

Accusations see the tract “anthropomorphizing” AI characters and that the platform’s chatbots connection “psychotherapy without a license.” Character.AI houses intelligence health-focused chatbots similar “Therapist” and “Are You Feeling Lonely,” which Setzer interacted with.

Garcia’s lawyers punctuation Shazeer saying successful an interview that helium and De Freitas near Google to commencement his ain institution due to the fact that “there’s conscionable excessively overmuch marque hazard successful ample companies to ever motorboat anything fun” and that helium wanted to “maximally accelerate” the tech. It says they near aft the institution decided against launching the Meena LLM they’d built. Google acquired the Character.AI enactment team successful August.

Character.AI’s website and mobile app has hundreds of customized AI chatbots, galore modeled aft fashionable characters from TV shows, movies, and video games. A fewer months ago, The Verge wrote astir the millions of young people, including teens, who marque up the bulk of its idiosyncratic base, interacting with bots that mightiness unreal to beryllium Harry Styles oregon a therapist. Another caller study from Wired highlighted issues with Character.AI’s customized chatbots impersonating existent radical without their consent, including 1 posing arsenic a teen who was murdered successful 2006.

Because of the mode chatbots similar Character.ai make output that depends connected what the idiosyncratic inputs, they autumn into an uncanny vale of thorny questions astir user-generated contented and liability that, truthful far, lacks wide answers.

Character.AI has present announced several changes to the platform, with communications caput Chelsea Harrison saying successful an email to The Verge, “We are heartbroken by the tragic nonaccomplishment of 1 of our users and privation to explicit our deepest condolences to the family.”

Some of the changes include:

“As a company, we instrumentality the information of our users precise seriously, and our Trust and Safety squad has implemented galore caller information measures implicit the past six months, including a pop-up directing users to the National Suicide Prevention Lifeline that is triggered by presumption of self-harm oregon suicidal ideation,” Harrison said. Google didn’t instantly respond to The Verge’s petition for comment.