Child AI Safety

Vals AI has launched an initiative to develop independent evaluations of how AI systems handle mental-health-related interactions with children and teenagers, citing rising youth use of chatbots and companion AI alongside concerns about potential harms. The effort follows research indicating that many young people seek mental health advice or companionship from AI, legal cases alleging chatbot-related harm to minors, and studies showing that models can fail in longer or nuanced conversations despite performing well on direct safety prompts. Vals is working with clinicians and academic researchers to assess self-harm and crisis responses, inappropriate assumptions of therapeutic or authoritative roles, and unhealthy emotional dependence, with an emphasis on how risks develop across multi-turn exchanges. Early findings suggest that models generally recognize explicit suicide disclosures but vary significantly in the usefulness of their follow-up, while more subtle risks can emerge through ordinary questions about health, treatment, or support and through gradually developing relational dynamics. The initiative comes as AI companies introduce youth-focused safeguards and policymakers pursue reporting, risk-assessment, and independent auditing requirements, and Vals plans to share detailed findings with policymakers before publishing further methodology and results.

No tracked trend matches for this post yet.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.