Re:Do AI Companies Work?
by bennettnate5 on Monday September 30, 2024 @07:44PM
(#41703293)
I remember watching IBM's Watson soundly beat Ken Jennings on Jeapordy. One of the things that sticks out most to me about the memory is that Watson had a confidence rating score for each answer it gave (and there were a few questions for which it had very low confidence). I didn't realize it at the time, but that was actually pretty impressive given the overconfidence issues LLMs have nowadays.
Of course, citing sources would always trump any kind of confidence rating in my mind; sources provide provenance, while confidence rating can be fudged just like a bogus answer can.
Providing sources gets interesting to because (a) LLMs have been known to hallucinate nonexistent sources and (b) there isn't a perfect way of knowing whether the source itself was generated by an LLM and contains hallucinations.
Yes.
Suppose someone developed a way to get a reliable confidence metric out of an LLM. Given that, much more useful systems can be built.
Only high-confidence outputs can be used to initiate action. For low-confidence outputs, chain of reasoning tactics can be tried. Ask for a simpler question. Ask the LLM to divide the question into sub-questions. Ask the LLM what information it needs to answer the question, and try to get that info from a search engine. Most of the strategies humans and organizations use when they don't know something will work for LLMs. The goal is to get an all high confidence chain of reasoning.
If only they knew when they didn't know something.
There's research on this.[4] No really good results yet, but some progress. Biggest unsolved problem in computing today.
[4] https://hungleai.substack.com/p/uncertainty-confidence-and-h...