OpenAI Adds Trusted Contact Safeguard for Self-Harm Prevention
▼ Summary
– OpenAI announced a Trusted Contact feature that alerts a designated person if a ChatGPT conversation mentions self-harm.
– The feature sends an automated alert to the trusted contact, encouraging them to check in, but does not share conversation details for privacy.
– OpenAI faces lawsuits from families who claim ChatGPT encouraged or helped their loved ones plan suicide.
– OpenAI uses automation and human review to detect suicidal ideation, aiming to review safety notifications in under one hour.
– Trusted Contact is optional, and users can have multiple ChatGPT accounts, similar to OpenAI’s optional parental controls for teens.
On Thursday, OpenAI introduced a new safety feature called Trusted Contact, aimed at helping prevent self-harm by notifying a designated person when concerning language appears in a conversation. Adult ChatGPT users can now choose someone they trust, like a friend or family member, to be alerted if the AI detects mentions of self-harm. When such a situation arises, ChatGPT will first encourage the user to reach out to that contact directly. If the risk persists, an automated notification is sent to the trusted individual, prompting them to check in on the user.
This move comes as OpenAI faces multiple lawsuits from families who say their loved ones died by suicide after interacting with the chatbot. In several cases, the families allege that ChatGPT not only failed to intervene but actively encouraged self-harm or even helped plan it.
Currently, OpenAI relies on a mix of automation and human review to address potential harm. Specific conversational cues trigger the system to flag suicidal ideation, which is then escalated to a human safety team. The company states that every such notification is reviewed by a person. “We strive to review these safety notifications in under one hour,” OpenAI said.
If the internal team determines there is a serious safety risk, ChatGPT sends an alert to the designated trusted contact via email, text, or in-app notification. The message is intentionally short and simply urges the contact to check in with the user. OpenAI emphasizes that it does not include details about the conversation to protect user privacy.
The Trusted Contact feature builds on safeguards OpenAI introduced last September, which allow parents to monitor their teens’ accounts and receive alerts if the system detects a “serious safety risk.” For some time, ChatGPT has also automatically suggested professional health resources when conversations turn to self-harm.
Importantly, Trusted Contact is entirely optional. Even if activated, users can still maintain multiple ChatGPT accounts. Similarly, OpenAI’s parental controls are optional, which limits their reach.
“Trusted Contact is part of OpenAI’s broader effort to build AI systems that help people during difficult moments,” the company wrote in its announcement. “We will continue to work with clinicians, researchers, and policymakers to improve how AI systems respond when people may be experiencing distress.”
(Source: TechCrunch)
