> It’s seductive to think AGI will suddenly advance technology dramatically, but dumping more effort into existing technology has a bad habit of hitting diminishing returns. Ideas like nano-machines seem to have vast potential, until you realize biology is already operating at those scales and has significant limitations.

There are a few reasons to think that AGI will significantly outproduce humans; it's likely cheap to run compared to the training costs. GPT-3 can generate text quite a bit faster than a human at maybe a billionth of the cost of its training, for example. Once achieved it's likely that practical progress on a lot of problems will actually be pretty rapid. Even if we only achieve parity with human intelligence and can never improve AGI beyond that we'll experience a few orders of magnitude in speedup of research.

I think it's a little early to write off nanotechnology when we haven't seen much more than tech demos at this point. Biology is a soup of proteins using brownian motion to find reaction sites. Highly impressive and amazing to be sure but needs to be wet, fed, and in a narrow temperature band. Precision molecular engineering is a game-changer in so many ways.

Producing text more cheaply is a poor metric for technological advancement. Going to the basics of say energy production, heat engines are already close to theoretical limitations. We already have 60+% efficient power plant so nothing can double future efficiency, and even in theory hitting 100% is never going to happen. The world has better technology and more infrastructure than 10 years ago, but it’s almost entirely evolutionary not revolutionary. Even the tantalizing self driving cars in the near future come with a massive downside of congestion from cars driving without people in them.

As to nanotechnology, we already design molecules to interface with biology via synthetic drugs, so essentially biological nanotechnology is going to be about creating larger structures not smaller ones.