... well, this is functioning as intended. It's a next-word prediction model.

Jesus Christ you need to stop spamming the same tokenized response to every comment on this thread or at least add a damn GOSUB routine.

Twice is hardly spamming. But whatever, I have no intention of repeating myself.

This argument is highly unsatisfactory. The only way for humans to divide two large numbers is to perform long division. That doesn’t mean that’s how a computer does it.

GPT LLMs are literally highly-skilled probabalistic next word predictors. It would be an astonishingly bad GPT that could not unscramble 12 words into a meaningful sentence. I just don’t think success at that task tells us anything other than it is a GPT.

If computers calculated arithmetic by prediction or inference then it might well be how they do it.

You can come to your own conclusions. The point of the above comment is simple. Either way, Blind probability is not enough. Not even close.

I genuinely don't know what you mean by "blind probability" or "surface-level probability", but in case you don't know, how transformer models work is after tokenization and embeddings are calculated each attention head in the model calculates "self-attention" which is the relationship between all tokens in the current context of the model and passes its results either to another layer of attention heads or to the decoder layer of the model. This typically uses a temperature parameter to control randomness in the output by affecting the probability distribution of tokens in the vocabulary when producing output. I'm not sure whether this counts as "blind" or "surface-level" but that's the probability in GPT models. If you want to learn more, here's Vaswani et al 2017 - the seminal paper https://arxiv.org/pdf/1706.03762.pdf