AI matched human mental-health judgments in 80% of tests

A Reddit thread can begin with a symptom and end somewhere deeper: a family conflict, isolation or another root cause. At Victoria University, Associate Professor Khandakar Ahmed and Saima Rani spent two years teaching AI to look for those themes across one million health-related Reddit threads. In tests on new posts, PaLM 2 matched human judgments in 80% of cases.

The team first built a mental-health narrative database. Psychiatrist and University of Melbourne honorary associate professor Manjula O’Connor manually classified 800 threads, linking the language people used to root-cause categories from Healthdirect Australia. That expert-verified data was then fed into two AI models, PaLM 2 and AutoML, which evaluated new threads separately from human raters.

The results were not identical. AutoML matched the human raters in 68% of cases, compared with PaLM 2’s 80%. The models were not simply checking for a list of symptoms: the researchers tested whether they could identify deeper emotional themes and possible causes in people’s own words. The paper appears in JMIR AI.

In practical terms, the method could give mental-health services another way to sort incoming narratives, support screening and triage, onboard patients, review their stories or power digital services. That could help professionals prioritize attention and personalize the first step in care, but the study presents those uses as possibilities. It does not establish a deployed clinical system, and the researchers acknowledge limitations.

The finding is therefore a measured one: an AI model aligned with human judgment in four out of five test cases after training on expert-verified material. Ahmed’s team has shown a research pathway for reading large volumes of personal narratives; health-care organizations would still need to determine how such systems perform beyond this evaluation and how they fit into real patient care.