News for hackers. Stuff that matters.
Re:Ilya Sutskever to leave OpenAI by sebastiennight on Wednesday May 15, 2024 @12:05PM (#40368765)
How many letters in the following sentence?

"هذا دليل سريع على أنه حتى البشر الأذكياء لا يمكنهم قراءة ”الرموز“ أو ”الحروف“ من لغة لم يتعلموها."

Re:Ilya Sutskever to leave OpenAI by omeze on Wednesday May 15, 2024 @04:07PM (#40371727)
I counted very quickly but 78? I learned arabic in kindergarten, im not sure what your point was. There are arabic spelling bees and an alphabet song just like english

The comment you replied to was saying LLMs trained on english cant count letters in english

Re:Ilya Sutskever to leave OpenAI by sebastiennight on Thursday May 16, 2024 @04:26PM (#40382727)
LLMs aren't trained in English with the same granularity that you and I are.

So my analogy here stands : OP was trained in "reading human language" with Roman letters as the basis of his understanding, and it would be a significant challenge (fairly unrelated to intelligence level) for OP to be able to parse an Arabic sentence of the same meaning.

Or:

You learned Arabic, great (it's the next language I want to learn so I'm envious!). But from the LLM point of view, should you be considered intelligent if you can count Arabic letters but not Arabic tokens in that sentence?

Re:Ilya Sutskever to leave OpenAI by lewhoo on Wednesday May 15, 2024 @06:34PM (#40373219)
Is this even a fair comparison ? Are we asking a LLM to count letters in an alphabet it never saw ?
Re:Ilya Sutskever to leave OpenAI by truculent on Wednesday May 15, 2024 @07:30PM (#40373586)
Yes, it sees tokens. Asking it to count letters is a little bit like asking that of someone who never learned to read/write and only learned language through speech.