It isn't just the computer power that allows us to be intelligent. It's also the bandwidth (which is always an order of magnitude or more behind the computer power) and the algorithm (which we don't have a clue how to create).
State of the art visual processing that gets so touted in the press is brittle -- it has to see very similar examples or it will fail. Neural networks don't transfer to new problem domains well at all.
Neural Networks have no sense of self or agency and they never will. There are key parts missing (like the ability to experiment with the environment). I'm not saying we will never have general intelligence, just that it's quite a ways away and the algorithms will be significantly different to the neural networks we use today. That said there will probably be many recognizeable components, like backpropagation, recurrent nodes, bayesian estimations, etc.
The issue with physical structures is that eventually the mass and stresses in the macro structure overcome the strength in the micro-structure. That is why nature stops at, eg, elephants.
It isn't obvious that intelligence suffers from such limits, as the only time the limit of intelligence has been tested (in terms of evolution) was when humans tried it. There is no evidence humans are pushing the limits of what intelligence can achieve. Quite the reverse, honestly, when you look at the performance of computers so far.