Our visual senses aren't camera feeds - the image we are aware of isn't raw input from the eyes, but rather it's generated out of some combination of raw inputs, low-level hacks to hide limited FOV and discontinuities caused by saccades, and higher-level interpretations based on how we feel, what seems to fit the context based on our prior experiences, etc.
I think it makes sense to assume that this process has many extra off-ramps that can trigger high-level reactions while bypassing conscious awareness. I.e. imagine that half-way through the post-processing stack, the relevant parts of the brain are somewhat convinced you're looking at a fast-moving object about to hit you in the face. There's a point at which it makes sense for the brain to make your body start a dodging move, even if it may be a misprediction, rather than waste a few dozen milliseconds to make sure, at the risk of getting a ball (or a rock, or a fist) up your nose.
Yes, one trick is that. Our computers have to serialize everything. We don't know how to build or program massively parallel computers with distributed memory. But the GP already touched on that point.
Human brain has to do a lot of work processing and integrating all the senses we use, which could cover at least for part of the neural capacity you mention, but I feel that where thinking is concerned, we may have accidentally solved the core problems. I somewhat expect that we'll look back and see that current LLMs were to human minds what the ol' Model T is to Tesla Model S - vastly different in levels of advancement, but also fundamentally the same thing.
Another excellent point. The idea that we may stumble onto massive computational leaps based on imprecise data processing requirements is terrifying. I do think we are nowhere near where we will need to be with respect to computation and, more important, information throughput. But the fact that we "oops"-ed into such a strong statistical model of conversation is deeply unsettling. I do believe GPT based language models will be part of an AGI.
Now, it would be one hell of a coincidence if the model we "oops-ed" into with LLMs, and the model evolution "oops-ed" into, were completely unrelated approaches to the same problem.
For our own brains, this is because evolution is the OG incremental learner - it just doesn't do leaps of faith, nor does it invest resources. Incremental evolutionary advancements must quickly stack into something improving survival rates, or else they get selected out of the gene pool. This strongly suggests that brains (human and animal alike) are based on a design that scaled easily and continuously - one that started simple and could be improved step by step, with each step being simple and conferring some non-zero survival advantage.
For our AI work, LLMs are exactly this: they derive from simple models that were iterated on over the past few decades, in small steps. They are still structurally simple as programs - but they've hit the right structure that allowed them to make qualitative jumps in capabilities from simple scaling - more parameters, more memory, more compute, more training time, more input data. This may be just my own perception, but I feel that LLMs are exactly the kind of model that an evolutionary process would be able to develop incrementally.
With the above in mind, plus the fact that LLMs are trained on the output of our brains (and their output is rated by our brains), makes me believe it's highly likely that, with language models, we've stumbled on the very same fundamental method of implementing thinking that is core to how our own brains work.
A more complex decision such as swerving to avoid a wheel that just came off the car in front, probably takes around ~500 milliseconds, or 50 computational steps.
Considering that modern computers need billions of computational steps to start a calculator app and draw it on the desktop, we have a long way to go.
Though there has been regression on this point, a NeXTStation Color could do the same with ~50 millions cycles on a 1120 × 832 12 bit display. (on 12 MB of RAM and 1.5 MB of VRAM.)