In this context, if we assume that Deep Thought from Hitchhiker's Guide is an LLM, then the answer to everything[1] i.e. 42 makes sense. 42 is just the token id !
1. https://en.m.wikipedia.org/wiki/Phrases_from_The_Hitchhiker%...
That was my theory as well when I first saw the strawberry test. However, it is easy test if they know how to spell.
The most obvious is:
> Can you spell "It is wonderful weather outside. I should go out and play.". Use capital letters, and separate each letter with a space.
The free tier ChatGPT model is smart enough to understand the following instructions as well which shows that its not just the simple words:
> I was wondering if you can spell. When I ask you a question, answer me with capital letters, and separate each word with a space. When there is real space between the letters, insert character '--' there, so the output is easier to read. Tell me how the attention mechanism works in the modern transformer language models.
Also somebody pointed out in some other HN thread that the modern LLMs are perfect for dyslexic people, because you can typo every single word and the model still understands you perfectly. Not sure how true this is, but at least a simple example seems to work:
> Hlelo, how aer you diong. Cna you undrestnad me?
It would be interesting to know if the datasets actually include spelling examples, or if the models learn how to spell form the massive amount of spelling mistakes in the datasets.
Why can't gpt pick up a non-fundamental "understanding" of letters and spelling from the data?
I mean... I do think "see letters" either when speaking/hearing, but I do know how to pull up those letters when necessary.
Why are other LLMs able to do it? (Other comments show images successfully generated with grok and flux.1)
"Beavers mate for life, 11 > 4"
How do you find this out?
pip install tiktoken
>>> import tiktoken
>>> encoding = tiktoken.encoding_for_model("gpt-4o")
>>> print(encoding.encode("hello marcellus"))
[24912, 2674, 10936, 385]