> No you "know" what your name is and you retrieve that information.
The LLM has this (literally - most of it is a key value store). It outputs a probability for all possible next tokens.
> It's infinitely different to rolling a dice and picking a name based on what number comes up
The sampling algorithm running the LLM then does this part, and adds randomness to make it more "creative".
So if you want factual information then don't be so random you skip over the right answer.
While the LLMs do have hallucinations, basic stuff like this will never trigger any.
Where I feel the most differences are aren't in the token concept but rather on the deep reasoning (which we don't use as much in my opinion)
If you do beam search, RAG, tool usage, etc then the whole system no longer is one.
This line of reasoning that LLMs "only predict" the next token is akin to saying humans can only think or speak one word at a time. Yes, we use one token/word at a time, but it is the aggregate thought that matters, regardless of what underlies it.
If there are 50K possible tokens and I don't have any other information, I could make a naive estimate that every token has equal probability and start generating text that is just gibberish. With the simple single-token Markov-chain example I would estimate probabilities based the previous token, and that probability estimate would be much better. If you use it for generating text it will look like something that is almost, but not quite, entirely unlike human speech. [1]
The difference lies entirely in how accurately you model the world and what information you have available when estimating probabilities. Models like GPT4 happen to be very good at it because they encode a huge amount of knowledge about the world and take a lot of context into account when estimating the probability. That's not something to be taken lightly.
These two things cannot be compared or contrasted. It's very common to see people write something like "LLMs don't actually do <thing they obviously actually do>, they just do <dismissive description of the same thing>."
Typically, like here, the dismissive description just ignores the problem of why it manages to write complete novel sentences when it's only "guessing" subword tokens, why those sentences appear to be related to the question you asked, and why they are in the form of an answer to your question instead of another question (which is what base models would do).