Good points - mechanically just following algorithms isn't thinking, and neither is "predicting the next tokens".
But would a combination of the 2 then be close to what we define as thinking though?
But would a combination of the 2 then be close to what we define as thinking though?
Why do you think that this is the part that requires intelligence, rather than a more intuitive process? Because they have had machines that can do this mechanically for well over a hundred years.
There is a whole category of critiques of AI of this type: "Humans don't think this way, they mechanically follow an algorithm/logic", but computers have been able to mechanically follow algorithms and perform logic from the beginning! That isn't thinking!