The whole reason there is an AI here is because a markov chain, which is what you are describing, doesn't work beyond one or two word horizons.
Not to mention that it doesn't just select which word it thinks is MOST probable, because that has been shown to lead to stilted and awkward output. Instead it randomly selects from the top few thousand possible words with probability based on the model's estimation
Would be nice to have enough of a theory of intelligence to be more precise than that, but the above definition will go very far.
Humans are very irrational but are still very good at this when they want to be but not always. A limiting factor for GPT4 is probably computing space/power.
AI can reason, it might not be the greatest especially at numbers and where there's data contamination but it can do it.
There's something called abductive reasoning, a gift and a curse at the same time.