First, AI is far more than just chatgpt, don't presume this is the same thing happening everywhere.
Second, The LLMs are all reasoning machines drawing on encyclopedic knowledge. A great example I recently heard is like a student parroting names of presidents to seem smart - it isn't thinking in the exact manner that we do but it is applying a reasoning to it. Chat GPT may be doing something akin to prediction, but it is doing it in a manner that is exposing reasoning. As the parent mentioned, our own brains use networks that refine over time with removal, and a huge number of our behaviors are "automatic". If you go looking for "consciousness" you may never find it in a machine, but it doesn't really matter if the machine can perfectly mimic everything else that you do or say.
An unfeeling unconscious yet absolutely "aware" and hyperintellgent machine is possibly the most alien we can fathom, and I agree there is no "end game" there is likely no mathematical limit to how far you can take this.