Re:A Jev that can ask Opus for help gives Opus accuracy at 4...
by locallylinear on Saturday November 07, 2026 @05:46AM
(#50403367)
> Calibration matters
I agree. What I’ve encountered though is even building enough cases for a calibration data set is hard. And so we farm out to smarter expensive models like Astra to label cases and use that as a calibration data set for the now cheaper LLM judge actually running as our judge
Is this standard ? Or am I just too lazy to manually label enough data to calibrate ?
I agree. What I’ve encountered though is even building enough cases for a calibration data set is hard. And so we farm out to smarter expensive models like Astra to label cases and use that as a calibration data set for the now cheaper LLM judge actually running as our judge
Is this standard ? Or am I just too lazy to manually label enough data to calibrate ?