Detecting when LLMs are uncertain

Re:Detecting when LLMs are uncertain by modeless on Friday October 25, 2024 @04:01PM (#41949087)

My understanding is that base models are reasonably well calibrated but the RLHF and other tuning that turns them into chat assistants screws up the calibration.

[ Parent | on HN ]

Re:Detecting when LLMs are uncertain by scottmf on Friday October 25, 2024 @06:52PM (#41950665)

There’s much that is lost but imo gpt-4-base would be borderline unusable for most of us compared to its descendants — perhaps even more so than GPT-3 davinci, at least relative to its time.

4 can be an absolute demonic hallucinating machine.

[ Parent | on HN ]