Putting aside that AI is terrible at novel discovery relative to its abilities otherwise, the direction of new technology is governed by incentives, not hypothetical good for humanity. Take social media as an example, which certainly had potential for good. How was it deployed? By which actors? To what end? And what were the unintended consequences? Not saying it’s all bad, but the utopia of the world becoming more connected and empathetic from the social-topological changes is clearly not the dominant outcome.
The material created will be a better poison/virus. The algorithm to keep the fusion tokamak from going boom will be at best 99% correct. The new solar cell? More exotic materials required than the current.
Blind optimism is a vice we cannot afford.
Reasonable is when you communicate the tradeoffs. More efficient, but more complex supply chain. Easier to immediately build, but we have to quadruple the computing power per cubic meter of reactor. More frequently discovered materials, but Uncle Ted can nail down the synthesis of the next VX, and a dispersal system in his garage.
Someone comes to me talking net gains. My next question is, "What's under the sheet you don't want to address out loud?" Under there is most often the real meat of the endeavor.
It's much easier to destroy and to enslave than to create something good. There I was thinking of the possible positive vs electricity cost.
Yes we might doom us all with AI. I hope the good win, wild time.
To look at the energy consumption of current model is extremely short sighted. If AI create a new material, a new solar cell, advance fusion reactor is all of humanity that jump forward.
Furthermore new generation of AI accelerators and new algorithms will improve efficiency by order of magnitute, it's still early days.