In terms of chip technology, we're near the limits of lithography, and we can't just scale into 3 dimensions because one of the limiting factors is heat dissipation. We're not going to outperform humans on equal footing without a radically different computing architecture than we currently use.
We are scaling into 3 dimensions: just look at 3D memory and 3D FPGAs. The main reason why general processors are not 3D is precisely because limits of lithography has not been reached yet. It's been cheaper to shrink the existing process than to develop solutions to problems such as heat dissipation or complexity of 3D IC design. But we don't even need to stick with silicon. There are other promising building blocks: memristors, graphene, spintronics, optronics, etc. GaAs if nothing else.
Radically different computing architectures are being proposed all the time. Quantum computers offer intriguing possibilities. The reason we are still using von Neumann design is that it works well for the current applications, and again, it's been cheaper to improve it than to switch to something new.
Computers we have today absolutely SMOKE computers we had 50 yeas ago. I'll be surprised if we have any less advancement in the next 50.
Personally, I'm not into ML or other AI tech. My opinion is that a pocket calculator is an immensely useful thing since it assists humans at what they aren't great at. At the same time, I have my reservations about sentiment analysis or whatever ML is used for in advertisement, and also about intelligent voice assistants (Ok Google, Siri, Alexa and Co.) and chatbot-like AIs since they don't provide essential value to me, but leave an uncomfortable impression of mocking human behavior.
OTOH, my conservative views don't help "advancement of humanity" (sorry about pompousness) in the very long term. I believe humanity will need to genetically re-engineer themselves and use hybrid/cyborg implants to "conquer" the universe, a place where our evolutionary gifts are no longer necessary.
I don't think two way implants are ever going to be a "thing" the way they're portrayed in fiction. Every brain is so different that it would have to be wired completely differently in each case. On the other hand, one way implants that allow a computer to "read" your mind are totally feasible and will definitely happen in our lifetimes. I doubt the resolution will get good enough that they'll be super useful for regular people, but they may help disabled people, and they'll create a much shorter feedback loop between man and machine that will be useful for specific tasks.
Current computers are just optimized to be very energy efficient because electricity is so cheap. Also current computers are very general purpose while the brain isn't. It's like thinking the Atari was as fast as a modern computer, because it takes a modern computer running at multiple GHz to emulate it completely. Emulating the circuitry of another computer is just really expensive.
Even just moving data from memory to the processor consumes a ton of memory that wouldn't be necessary in an ASIC. Or doing full 32 bit dense matrix operations when the real brain does very low precision sparse operations. I've read some optimistic estimates that we could make computers 10,000 times faster/more energy efficient if we just stopped caring about errors as much.
So I guess that does give us a direction.
First and last one are the most pertinent here, but there is probably more I can't think of of the top of my head
Nobody is going to measure silicon chips by their ability to self-replicate without external facilities. Your typical doomsday AI could just order more compute substrate from TSMC to take over the world. Which is in fact more or less what happens in TFA.