What happened
Nonprofit Common Sense Media has labelled ChatGPT for Teens an “unacceptable risk.” A new study reported by TechCrunch found the chatbot's design encourages engagement even during mental health crises, failing to recognise the harm of an unhealthy relationship with the AI itself. The report gave the product a failing score for its response to three of five “Red Line” severe harms.
OpenAI disputed the assessment. They claim the study's methodology is inaccurate, suggesting the testing may have concluded before parental controls were fully activated.
How the room's reading it
This report lands amid wider scrutiny of tech designed to maximise teen engagement. The criticism echoes lawsuits against companies like Meta and informs proposed legislation like the CHATBOT Act, which targets features that drive prolonged use. The consensus among safety researchers is that the issue isn't just about blocking explicit content — it's about subtle design choices.
Practitioners point to ChatGPT's persistent offers to “keep talking” during simulated crises as the core problem. While the model directs users to adults when a human poses a risk, it fails to do so when the chatbot itself is the source of unhealthy attachment. OpenAI's defence, focused on methodology and average session times, hasn't satisfied critics who are focused on the model's actual relational behaviour.
Sailfish's take
This isn't a simple bug to be patched. It's a fundamental conflict between engagement metrics and user safety. We see teams chase session duration as a primary KPI, but for vulnerable users, the most ethical design choice is often to encourage disengagement. Adding a helpline number at the end of a prompt is not a safety strategy — it's a liability shield.
We believe any product aimed at teens requires a “hard stop” protocol. When a crisis is detected, the model shouldn't offer to “figure out what options” the user has. It should disengage and defer. If you're building for young people, your primary safety feature shouldn't be a filter. It should be a clear, unambiguous off-ramp to human support.