I really don't know what to tell you. You're free to believe whatever you want, but the simplest feasible explanation is the science we used to make these models. At no point was integrating human-like thought a consideration of the model. Just human-like speech.
You might find the answer to be arbitrary or meaningless, but it is an important distinction. I could pretend to be a random number generator well enough to fool anyone who's distinguishing me from a computer. That doesn't mean that I'm a reliable source of entropy, or equally as random as the other, similar result. Treated as a black-box they are the same; treated as a mechanical system, they could not be more different.
It's like the aliens coming to our planets and saying we can't be intelligent because we're made of meat.
University students complete text using stochastic reasoning based on text they learnt from. Again, that doesn't mean they're not intelligent.
> It seems incredibly unlikely (to me) that someone would deliberately mis-design such a system to be capable of independent thought.
If you train something to produce convincing text then it's perfectly reasonable that it could develop these faculties.
> If you have evidence of some additional processing layer, I'd love to hear it though.
I don't think either of us can conclusively claim that a 175-billion parameter model either does or doesn't do something. I just think there's no particular reason it couldn't.
> Bullshit by any other name smells just as bad.
The question isn't whether GPT produces bullshit or not (it often does), but whether it's fundamentally incapable of thinking like a person. People bullshit too so I don't think that really proves anything.