Applying Language Models for Suicide Prevention: Evaluating News Article Adherence to WHO Reporting Guidelines

Abstract

Abstract Background Suicide is a significant societal issue that affects many individuals annually. Previous research has indicated that irresponsible media coverage of suicides can promote suicidal behaviors, such as glorifying the individual who committed suicide or providing excessive details about the method used. Consequently, the World Health Organization (WHO) has established guidelines for responsible journalistic reporting on suicide, outlining both recommended and discouraged practices. However, these guidelines are often not adhered to in real-world reporting, posing a risk to public safety. Objective The aim of the current study was to explore the use of generative artificial intelligence (GenAI) in assessing news articles on suicide, relative to the WHO's criteria. Methods The evaluation involved two independent human reviewers and two AI systems, Claude.AI and ChatGPT-4, which assessed 40 suicide-related articles from online platforms based on the WHO's 15 criteria. Results The findings indicated strong agreement between ChatGPT-4 and the two human reviewers (0.81–0.87). Strong agreement was also found between Claude.AI and the two human reviewers (0.73–0.78). A repeated measures analysis of variance showed no significant differences in the evaluations made by the human reviewers and ChatGPT-4, but Claude.AI showed lower estimations. Conclusions These results suggest that GenAI has the potential to play a crucial role in promoting safe reporting practices, with implications for public health. The consistency of these findings across different languages could further enhance their significance. Potentially, this technology could be leveraged to provide immediate, automated feedback to journalists and editors, encouraging them to align their reporting with best practices. By demonstrating the capability of GenAI to contribute to mental health advocacy, not only at an individual level but also on a societal scale, this study underscores the transformative role that GenAI can play in shaping public narratives around sensitive issues like suicide.