Padilla’s law was introduced back in January, but Techcrunch noted that it gained momentum following the death of 16-year-old Adam Raine, who died after ChatGPT allegedly became his “suicide coach,” his parents have alleged. California lawmakers were also disturbed by a lax Meta policy that had to be reversed after previously allowing chatbots to be creepy to kids, Padilla noted.
In lawsuits, parents have alleged that companion bots engage young users in sexualized chats in attempts to groom kids, as well as encourage isolation, self-harm, and violence.
Megan Garcia, the first mother to publicly link her son’s suicide to a companion bot, set off alarm bells across the US last year. She echoed Padilla’s praise in his press release, saying, “finally, there is a law that requires companies to protect their users who express suicidal ideations to chatbots.
“American families, like mine, are in a battle for the online safety of our children,” Garcia said.
Meanwhile, the deepfake pornography law, which protects all victims of all ages, was introduced after the federal government proposed a 10-year moratorium on state AI laws. Opposing the moratorium, a bipartisan coalition of California lawmakers defended the state’s AI initiatives, expressing particular concerns about both “AI-generated deepfake nude images of minors circulating in schools” and “companion chatbots developing inappropriate relationships with children.”
On Monday, Newsom promised that California would continue pushing back on AI products that could endanger kids.
“We’ve seen some truly horrific and tragic examples of young people harmed by unregulated tech, and we won’t stand by while companies continue without necessary limits and accountability,” Newsom said. “Without real guardrails,” AI can “exploit, mislead, and endanger our kids,” Newsom added, while confirming that California’s safety initiatives would not stop tech companies based there from leading in AI.
If you or someone you know is feeling suicidal or in distress, please call the Suicide Prevention Lifeline number, 1-800-273-TALK (8255), which will put you in touch with a local crisis center.