Re:New benchmark shows top LLMs struggle in real mental heal...
by zeroonetwothree on Wednesday December 10, 2025 @10:43AM
(#46218996)
Human therapists are often quite bad as well. It took me around 12 before I found a decent one. Not saying that LLMs are better but they do theoretically have more uniform quality.
[ Parent
| on HN ]
Re:New benchmark shows top LLMs struggle in real mental heal...
by RicardoRei on Wednesday December 10, 2025 @10:46AM
(#46219062)
Exactly. We don't do claims about humans. But there is room for improvement on current LLMs... For researchers to be able to improve LLMs we first need to know how to evaluate them. We can only improve what we can measure so we studied how to measure them :)
Re:New benchmark shows top LLMs struggle in real mental heal...
by arisAlexis on Wednesday December 10, 2025 @05:28PM
(#46224835)
So it's a cyclical grading? Like elementary math students grading each other's solutions right?
How can this even be valid scientifically
Re:New benchmark shows top LLMs struggle in real mental heal...
by nyeah on Wednesday December 10, 2025 @10:45AM
(#46219041)
Sure, uniformly zero as far as anyone knows.
Re:New benchmark shows top LLMs struggle in real mental heal...
by MengerSponge on Wednesday December 10, 2025 @04:17PM
(#46223988)
Quality is variable, but did any of those 12 encourage you to kill yourself?
If a therapist found to encourage any of their patients to self-harm would lose their license to practice and would likely face prosecution. The plagiarism machine should face the same level of scrutiny.
Re:New benchmark shows top LLMs struggle in real mental heal...
by mrguyorama on Wednesday December 10, 2025 @04:26PM
(#46224110)
How many were "therapists" or "Counselor" vs actually credentialed professionals?
There's also a lot of credentialed professionals who got their credential decades ago and haven't at all kept up with the significant changes or new data over that time. This is a pretty big problem in all of medical care.