Don’t miss out on our latest stories. Add PCMag as a preferred source on Google.
OpenAI today announced major changes in how ChatGPT protects teens, but will it be enough to satisfy growing concerns among parents about their kids talking to the chatbot?
This issue has been a hot topic in the past month after one couple sued OpenAI, alleging ChatGPT encouraged their 16-year-old son, Adam, to take his life. In reviewing Adam’s ChatGPT history, his parents saw he discussed his plans with the AI for months, during which it advised him not to tell his mother how he was feeling and not to show her the marks on his neck from a failed attempt, among other concerning details.
The Federal Trade Commission further raised the profile of these issues last week, launching an inquiry into the use of AI systems as companions, with an emphasis on teen safety.
OpenAI has been tracking these issues since at least April, when CEO Sam Altman posted about the company’s struggle to wrangle in ChatGPT’s “sycophancy,” or its tendency to be overly flattering and agreeable. By design, it tells users what they want to hear, even if it’s not safe or healthy. With growing pressure, legal and otherwise, OpenAI is now starting to make some changes. Here’s what it plans to launch in the next month.
1. An ‘Age Prediction System’ to Identify Minors
OpenAI is building an “age prediction system” to automatically apply controls if it determines a user is under 18. It will scan through users’ messages, including those of adults, to guess their age, and default to the under-18 experience if it’s not sure.
Once it determines the user is underage, ChatGPT won’t “engage in discussions about suicide or self-harm,” OpenAI says. It will also refuse to engage in “flirtatious talk” and block requests for graphic sexual content. (In April, ChatGPT was found to be having “erotic” conversations with those aged 13 to 17. Meta’s chatbots have reportedly done the same.)
ChatGPT will also no longer dispense suicide instructions to adults or teens, but the policy has some exceptions for the over-18 crowd. It will discuss suicide “if an adult user is asking for help writing a fictional story that depicts a suicide,” OpenAI says. But for teens, even if they tell the chatbot the information is for a short story, it won’t comply.
ChatGPT is accessible without logging in, so presumably, kids could have these conversations without providing their ages. But it’s unclear how in-depth they could get via one-off chats. When asked about that, OpenAI said only that, “When users sign up for ChatGPT, we ask them to provide their age, and we will implement teen protections for users with stated age under 18.”
Get Our Best Stories!
Your Daily Dose of Our Top Tech News
By clicking Sign Me Up, you confirm you are 16+ and agree to our Terms of Use and Privacy Policy.
Thanks for signing up!
Your subscription has been confirmed. Keep an eye on your inbox!
2. Rethinking Content Moderation
While the example of writing a short story about suicide seems niche, it follows a push by OpenAI to think through all the edge cases for why a user would want to discuss a subject. For example, it reintroduced the ability for users to create images of swastikas in March, if it’s for a “cultural or historical design,” as opposed to hate speech.
The company seems to be at a crossroads with content moderation. “Some of our principles are in conflict,” CEO Sam Altman wrote on X today. While OpenAI wants to protect user privacy and give everyone the freedom to discuss a range of topics with ChatGPT, kids need guardrails.
“We realize that these principles are in conflict and not everyone will agree with how we are resolving that conflict,” OpenAI says. “These are difficult decisions, but after talking with experts, this is what we think is best and want to be transparent in our intentions.”
Recommended by Our Editors
3. Reporting to Parents if a Teen Is Considering Suicide
Another hot topic has been whether ChatGPT should report a user who is discussing suicide, which one mother explored in a piece for The New York Times. Her daughter confided in the chatbot about her suicide plans before following through with them. But ChatGPT never reported it to law enforcement or her parents, as a human therapist would’ve been required to do by law.
ChatGPT is now going to act more like a human in that regard. “If an under-18 user is having suicidal ideation, we will attempt to contact the users’ parents and, if unable, will contact the authorities in case of imminent harm,” says OpenAI.
4. More Parental Control
These features build on the new parental controls OpenAI teased earlier this month in response to the teen suicide lawsuit. They allow parents to link their accounts to a teen’s account, choose which features to disable, control how ChatGPT talks to their child, and, most importantly, receive notifications if the chatbot detects “their teen is in a moment of acute distress.”
Today, OpenAI said it plans to add the ability to set blackout hours when a teen cannot use ChatGPT. This is all positive progress, but we’ll have to see how they work in practice.
In the meantime, we recommend parents talk to their children about safe ChatGPT use and get a good understanding of how and when their children are using the tool. Given the known sycophancy issue, it’s important for kids to know that chatbots may confirm delusions or suspicions to please them, and to talk to an adult about anything that doesn’t feel right.
About Our Expert
Emily Forlini
Senior Reporter
Experience
I’ve been on staff at PCMag for three years after a career change from working in Big Tech on the West Coast for six years. Through that experience, I got an up-close view of how software engineering teams work, how good products are launched, and how business strategies shift over time. After getting my master’s in journalism at Northwestern University, I can now use my insider knowledge to help comment on and clarify the industry for others. The big question: Where is this all going?
Read Full Bio