News for hackers. Stuff that matters.
Re:AI-enhanced development makes me more ambitious with my p... by spyder on Friday March 31, 2023 @08:03AM (#35385758)
Another trick I have found interesting and somewhat useful: asking it to give you his confidence score for it's statements, and a lot of the time his confidence are lower for his wrong statements but also lot of the times has high confidence in some wrong answers too (for example gets simple math wrong with 100% confidence), so not sure it's really that useful just an interesting thing to try.
Re:AI-enhanced development makes me more ambitious with my p... by gbasin on Friday March 31, 2023 @10:39AM (#35387844)
The RLHF process performed on that chat models actively confuses their ability to estimate confidence. The raw models are quite well calibrated though