Have you read the GPT-3 paper? The authors are pretty forthright about how the GPT techniques likely won't scale much beyond 175B parameters.

Yet, the darn thing still can't reason.

They "can't reason" but they can solve symbolic integration and differential equations (https://arxiv.org/pdf/2006.06462.pdf), beat us at all board games, and more recently get to be almost impossible to tell apart from human writing (GPT3).

Five years ago we would have said "the darn thing still can't write a cohesive paragraph".

That's still true. It can write a paragraph that's usually grammatical, and can stay on topic, but it's missing things like facts, or even the ability to remember which side of an argument it's taken previously.

I think over the course of several paragraphs that's true, but within one it tends to be pretty good.

People look at the present and feel like that's all that's there. You don't look at a 2-year old and say he barely makes any intelligible sounds yet.

To be fair, it wasn't trained to.

To be really fair, I think you need to more precisely define what you mean by "reason". It absolutely CAN reason by some measures.