News for hackers. Stuff that matters.
Re:Llama 2 is about as factually accurate as GPT-4 for summa... by sorokod on Tuesday August 29, 2023 @02:32PM (#37311944)
"It is not too much of a stretch to conclude that a system that is better at telling factual from non-factual sentences is better at not making them up in the first place – or alternatively could decide through a two stage process if it was being inconsistent."

Stretching aside, how does one follow from the other?

Re:Llama 2 is about as factually accurate as GPT-4 for summa... by alfalfasprout on Tuesday August 29, 2023 @02:40PM (#37312035)
They don't.
Re:Llama 2 is about as factually accurate as GPT-4 for summa... by waleedk on Tuesday August 29, 2023 @03:56PM (#37313051)
[Author] The problem is: how could you objectively evaluate the factuality in any other way? Summary ranking is an established approach in the literature.
Re:Llama 2 is about as factually accurate as GPT-4 for summa... by mullingitover on Tuesday August 29, 2023 @03:02PM (#37312313)
If the system can evaluate the truthfulness of a statement, and it can generate statements, then it can create arbitrary numbers of candidate statements and filter for truthfulness.
Re:Llama 2 is about as factually accurate as GPT-4 for summa... by sorokod on Tuesday August 29, 2023 @03:30PM (#37312707)
Looks like you take the "alternatively" fork in the original statemet - is it not a tautology?
Re:Llama 2 is about as factually accurate as GPT-4 for summa... by waleedk on Tuesday August 29, 2023 @03:55PM (#37313029)
[Author] This is what I meant. That even if it doesn't work at a single layer, you could build a "verifier" stage whose approach is similar to this verification process and it can make the decision.