With AI you can optimize or change the underlying algorithms, or allocate more computing resources as necessary. You can't do that with humans—at least not in any literal sense, nor in any way that is remotely as efficient or effective.

The very nature of human intelligence is such that its architecture, scale and speed of computation are all bound within a relatively narrow range.

Humans don't scale. If you try to add more of them, you get things like governments and markets, i.e. lots of people who can't agree on shit. Not very efficient. Scaling a properly designed AI, on the other hand, could lead to something much more powerful - cognitive power of great many humans with a clear focus of an individual.

I'm not sure why you think humans don't scale. Do you think that the research done by the Manhattan Project was no more impressive than research conducted by a single individual? Or that a single individual is going to have as much success gaining political power as a political party?

Now as I said, "adding another person doesn't necessarily make things go faster." It can, but not always. But what evidence do we have that a human level AI would be able to scale easily? That seems to be a pretty huge assumption that gets glossed over.

>But what evidence do we have that a human level AI would be able to scale easily?

The fact it would probably be software, executing on a digital platform. These things typically lend themselves to optimization and scaling.

Seems to me that the chance we'll have just enough computing power to run human-level AI—without sufficient resources to go beyond that—is a small one. It is far more likely that we will either have too little, or more than ample computing power available.

What evidence do you have that human-level AI won't scale easily?

The best argument I can think of in your favor would be prohibitive computational complexity requirements, but the aforementioned dynamic still applies—it's unlikely that we'd have only just enough computing power to achieve human-level and not go beyond that.

We can look at efforts to stimulate the brain, which do not seem to scale easy. As I wrote in another comment, the K computer was able to simulate a seconds worth of brain activity. It doesn't seem to have scaled easily. I don't have experience with supercomputers, but I have a feeling that things are a bit more complicated than "just network it together with another supercomputer, and then everything will be twice as fast."

And that's just talking about a raw increase in computing power. We don't have any idea whether a human level intelligence program would even be able to put increased processing power to good use (and it could theoretically be detrimental; look at what happens to humans who have too many neuronal connections).

>We can look at efforts to stimulate the brain, which do not seem to scale easy.

Simulating the brain isn't necessarily the path to human-level AGI though. Simulating biology in digital form tends to be inefficient.

Modern machine learning scales easily, or at least relatively easy.

>(and it could theoretically be detrimental; look at what happens to humans who have too many neuronal connections)

Perhaps a constraint of our brain's cognitive architecture and not intrinsic to cognition itself. Still, interesting point.

Human minds in meat don't scale. But upload, and make a bunch of copies, introducing more-or-less random variation. Run in parallel, with suitable aggregation and selection. Got that from Quantum Thief by Hannu Rajaniemi.

You shouldn't consider government inefficiency a flaw. That's by design. If you have a government or a brain that makes decisions quickly, you might not have time to backpedal on seemingly good ideas that turn out bad.

I was thinking more in terms of cost. Adding a person to your cause requires a lot of money (the cost of them devoting their entire working time), and you only get so much extra benefit (efficiency losses due to communication and their own incentives). And the more effective the human, the more he would cost.

With an AI, you just dump that money directly into the brain by slapping on more compute power and storage. It'd be a direct increase in ability to its will. I'm not sure what exactly the raw costs would be (maybe it would still cost more than hiring a human), but at least it seems like it'd be a much more efficient use of each dollar.

The supercomputer that just simulated one second's worth of brain activity (K computer) cost $1.25 billion to design and build and $10 million a year to operate. Now, I don't have experience with supercomputers, but I doubt that you'd just be able to slap in some extra ram and get a big speed up.

It seems likely that the first simulated human level intelligences, at whatever point in the future they pop up, will at first be extremely costly and extremely slow. We don't really know if, say, connecting two supercomputers two simulate twice as many virtual neurons would make it more effective or if it would make it too unstable to do anything (how would a human brain with twice as many neurons react? It might just be mentally unstable and unproductive). But even a simple doubling experiment like that would probably take a long time to plan out and realize.

This analogy would be accurate if humans had brain-to-brain interfaces. AI's that collaborate will be able to do so much faster than humans with their slow politicking. But in before that, an AI doesn't even need to collaborate with other entities, as it can expand its computing resources at will.