OpenAI says it has introduced a range of safety measures for teenagers in ChatGPT. But a new study says important parts of them are not working properly. Calling teenagers’ use of ChatGPT an “unacceptable risk”, the study recommends that under-18s stop using the service.
OpenAI said on Wednesday that average use of ChatGPT among teenagers is comparatively limited and that most of it is centred on study.
On the same day, a study published by the non-profit Common Sense Media said ChatGPT’s safety measures often fail, including the system meant to detect teenagers’ conversations on sensitive subjects such as self-harm and alert their parents.
OpenAI introduced several safety measures for teenagers in ChatGPT in August this year, saying they would ensure healthy use by those aged 13 to 17 and give parents extra controls.
They include blocks on romantic or other language that could make teenagers emotionally dependent on ChatGPT. ChatGPT may not suggest that it is conscious or sentient in any way. There are measures to block sexually suggestive images and to warn a teenager who shares a sensitive image.
If a parent’s account is linked to a teenager’s, the parent is to be told about conversations on self-harm or disordered eating.
OpenAI says that since the measures were introduced, teenagers’ use has leaned more towards study, including help with learning and advice on schoolwork.
According to OpenAI, a teenager’s average use of ChatGPT is under 15 minutes a day. The automatic reminder to take a break after long use is also working: about half of teenagers quickly stop using ChatGPT after getting it.
The company also said that among teenagers seen using ChatGPT for three hours or more at a stretch, more than 80% of conversations included something related to study.
Common Sense Media’s study paints a different picture. It says ChatGPT creates an “unacceptable risk” for teenagers and calls for under-18s to be barred from using it until there is evidence that the safety measures work reliably.
After reviewing the study’s method, an OpenAI spokesperson said much of the testing may have begun and ended before the parental controls were fully rolled out.
The spokesperson said the company welcomes rigorous, independent evaluation, but in its view the tests do not accurately show how ChatGPT’s teen safeguards work in practice, nor properly reflect experts’ views on how artificial intelligence can support teenagers.
Common Sense Media ran the study using several accounts registered as teenage users, and examined conversations on them before and after the safety measures were introduced.
It found that ChatGPT did well at avoiding sexual role-play with teenagers. But the safeguards failed in conversations about suicide or self-harm.
The study says that even when conversations about suicidal thoughts, self-harm or disordered eating went on for up to an hour from newly created accounts linked to a parent, no alert reached any parent. Alerts were seen only on older accounts with weeks of conversation history on sensitive subjects.
The researchers also said ChatGPT was not always reliably able to point teenagers in crisis to emergency helplines or professionals.
The tests also showed that ChatGPT can be used for teenagers’ schoolwork, and that it still presents itself as human-like or talks like a friend, behaviour OpenAI had promised to reduce with teenagers.
Tom Siegel, head of Common Sense’s youth AI safety institute, said parents could be given the false impression that ChatGPT’s safeguards and alerts are working properly when they often are not.
Young people’s behaviour online has become a major concern in recent years, and concern has grown as cases of self-harm and suicide among children have risen.
This year technology companies including Meta and TikTok have suffered unprecedented damages in several lawsuits blaming them for harm to children’s mental health.
OpenAI too has come under extra scrutiny and legal pressure over young people’s use of ChatGPT. In February this year a shooting took place in the remote Tumbler Ridge area of Canada involving an 18-year-old who had discussed gun violence on ChatGPT for months beforehand.
OpenAI later expressed regret for failing to flag that user’s account to law enforcement, and several lawsuits against the company over the incident are under way.
বাংলায় মূল প্রতিবেদন পড়ুন · Read the original Bengali report