News for hackers. Stuff that matters.
Re:Do large language models need sensory grounding for meani... by oezi on Monday March 27, 2023 @01:52AM (#35323311)
I also don't get why this line of reasoning doesn't take into account that LLMs can just emit a backspace token in the same way that humans can ask for you to discard their statement.

The probability of a correct answer doesn't have to decrease in the length of the answer.

Epistemologically it certainly makes sense that you need to run a randomized controlled trial at some point to ascertain facts about nature, but alas we humans very rarely do. We primarily consume information and process that.

Re:Do large language models need sensory grounding for meani... by Hendrikto on Monday March 27, 2023 @06:03AM (#35324786)
These LLMs are trained with lots and lots of text from many sources, 99.9+% of which do not contain backspaces. They could theoretically output them, given the right tokenizer, but they don‘t.

And they especially don‘t do it to correct errors.

Re:Do large language models need sensory grounding for meani... by cma on Tuesday March 28, 2023 @06:30PM (#35348927)
They could potentially be trained on applying diffs though. Wikipedia has full source control version history of how things evolved, and so do most of the source code corpuses they use like github.

Google has keystroke by keystroke history of all google docs, could pull an Adobe Stock and opt everyone in with some TOS update but the blowback would be huge.