However, I do not believe the same is true as a simulation of limited, superficial hallmarks of intelligence. That is what, based on my understanding of the concepts underlying ChatGPT and other LLMs, I believe them to be.
Sure, but that's a belief not a firm conclusion we can assert with anywhere near 100% confidence because we don't have a mechanistic understanding of intelligence, as I initially said.
For all we know, human "general intelligence" really is just a bunch of tricks/heuristics, aka your superficial hallmarks, and machine learning has been knocking those down one by one for years now.
A couple more and we might just have an "oh shit" moment. Or maybe not, hard to say, that's why estimates on when we create AGI range from 3 years to centuries to never.
Yes, but your assertion that this is not itself intelligence is based on the assumption that a simulation of intelligence is not itself intelligence. Intelligence is a certain kind of manipulation of information. Simulation of intelligence is a certain kind of manipulation of information. These may or may not be equivalent. Whether they are only time will tell, but we wary of making overly strong assumptions given we don't really understand intelligence.