News for hackers. Stuff that matters.
Re:New benchmark shows top LLMs struggle in real mental heal... by hoodsen on Wednesday December 10, 2025 @10:40AM (#46218950)
Do you have plans to improve the quality of the LLM as judge, in order to achieve better parity with human clinician annotators? For example, fine-tuning models? Thinking that the comparative clinician judgements themselves would make useful fine-tuning material.
Re:New benchmark shows top LLMs struggle in real mental heal... by RicardoRei on Wednesday December 10, 2025 @10:47AM (#46219101)
yep yep. Its something we have to study and its likely we can improve the LLM as a Judge further.

Same thing for the patient LLM. We can probably fine-tune an LLM to do a better job at simulating patients.

Those two components of our framework have space for improvement