News for hackers. Stuff that matters.

A comment on Why are LLMs general learners?. Parent comment

Re:Why are LLMs general learners? by rcme on Monday June 12, 2023 @10:26PM (#36304363)
It doesn't have anything to do with tokenization. You can define binary addition using symbols, e.g. a and b, and provide properly tokenized strings to GPT-4. GPT-4 appears to solve the arithmetic puzzles for a few bits, but quickly falls apart on larger examples.
[ Parent | on HN ]
Re:Why are LLMs general learners? by iliane5 on Tuesday June 13, 2023 @11:16AM (#36311490)
What I was saying is that because you need to go out of your way to make sure it's tokenized properly, I wouldn't be surprised if there are enough non properly tokenized examples in the dataset.

If that was the case, it would make it difficult to generalize these concepts.

[ Parent | on HN ]