Re:New benchmark shows top LLMs struggle in real mental heal...
by hoodsen on Wednesday December 10, 2025 @10:40AM
(#46218950)
Do you have plans to improve the quality of the LLM as judge, in order to achieve better parity with human clinician annotators? For example, fine-tuning models?
Thinking that the comparative clinician judgements themselves would make useful fine-tuning material.
[ Parent
| on HN ]
Re:New benchmark shows top LLMs struggle in real mental heal...
by RicardoRei on Wednesday December 10, 2025 @10:47AM
(#46219101)
yep yep. Its something we have to study and its likely we can improve the LLM as a Judge further.
Same thing for the patient LLM. We can probably fine-tune an LLM to do a better job at simulating patients.
Those two components of our framework have space for improvement