Tokenization as the main problem is a red herring. It's possible to get rid of the tokens entirely and train on byte sequences, it won't make a difference to why generative AI can't count or do basic arithmetic.