The difference is that it can use them not for sending low-return spam, but for multiplying its power - even if it doesn't give a qualitative difference, there should be a quantitative difference (how many different tasks it can do at once) when a strong AI gains many orders of magnitude more computing power than any laboratory/datacenter where it was initially created.

There's an assumption in most AI discussions that AI "must" be embarrassingly parallelizable. For example, much as my brain is parallelized, surely an AI would have to be the same way, much as all heavier than air human flight required was a faithful reproduction of bird anatomy (LOL).

There's also a certain assumption that intelligence implies a level of self awareness that most humans don't have therefore AI would be highly self aware...

There's also a whole industry of sophistry devoted to proving no humans are intelligent and IQ as a concept or any other numerical measure of intelligence does not exist and there is no way to compare intelligence, and right or wrong those folks will surely make life difficult for people improving / testing / upgrading an AI, either because they're right or they'll be making political protests.

But is it the right kind of computing power? The human brain is a very different sort of machine, architecturally speaking, from any computer currently in existence. Current computers have a very difficult time dealing with massive, parallel, random data access. Perhaps strong AI requires billions of threads randomly accessing a database which is terabytes in size? That is going to make the memory bus very, very sad.

You can find one smart person. But how do you find another smart person for them to work with? With AI-level smarts, you reduce the coordination problem to zero since it can spawn multiple copies of its own brain state (maybe?) and work concurrently and intelligently on the same problem. Just ignore the problem of killing off divergent brain states once the task is complete (nobody cares about the life of a kage bunshin).

I sure could get a lot more done if I had five more of me (unpaid, of course) to tackle a problem all at once.

The interesting question is: where is that upper limit on intelligence-per-FLOPS?

EDIT: Fixed it to "FLOPS". We already have AI algorithms that Solve The Problem if given unlimited "thinking time" between steps of interaction with their environment (up to the point of needing a Halting Oracle), and we have extensions of those that are asymptotically speed-optimal modulo some additive or multiplicative constants (which are larger than our current universe). I strongly suspect that scaling down these Idealized AI Agents to real, useful programs isn't going to just involve feeding raw instances of the Ideal Agents small enough environments to be tractable, but actually finding or deriving some algorithm that can scale continuously up and down with available computing power per unit of real time, where the time units are the actual real-time length of an interaction cycle.

To have a white-box learner would imply that humans were smart enough to build it. Yet it's unlikely that humans can build something significantly smarter than a human, so this AI would have the same limitation: It's unable (not smart enough) to build a smarter AI.

For instance, if you have infinite computing power, you just have to dovetail over every possible Turing Machine until one of them "wakes up" and does AI-y things. Yes, there are in fact real approaches based on a more sophisticated version of such a Universal Search.

In fact, we routinely make use of machine-learning algorithms that are "smarter than a human" in particular narrow domains. Are they smarter than a human at general intelligence, of the kind necessary to start looking more like a scifi "AI" than a cheap trick? No, but they are an indication that such a thing could be possible.

Overall, I would say that with Machine Learning maturing and Universal AI being a formal science at all, we are only just now beginning to find out where the actual tradeoffs lie in the design and construction of intelligent agents.

The speed of intelligence scales. But not the quality of it.

The quality of intelligence is something very hard to quantify. But as simply as I can state it, it's the quality of coming up with an idea that is not just an extension of an existing one, but an idea that came seemingly from nowhere. To someone else the idea appears magic, and no amount of thinking by that other person can lead him in a path that will end up there.

> but they are an indication that such a thing could be possible.

Real genius is not just "more" intelligence, it's intelligence of a different type that a non-genius simply can not do, no matter how they try. Analogously, just because a computer can do something fast does not give it that hard to define quality of true intelligence.

To make (rather than randomly search for) a computer with real intelligence we need to understand what it is, and where it comes from, and we don't know either one of those things. Our best (smartest) computers are all variations of "search randomly" (for example chess engines). Those can do some amazing things, but their limitations will not be overcome by doing more of the same.

> infinite computing power .. every possible

Talking about infinity is a total red herring and you know that - especially because if your method of increasing intelligence is "search randomly" then an AI will not be able to make a better AI - all it can do is continue the random search humans already started.

Well the good approach that requires a halting oracle is AIXI, which uses semi-approximable (IIRC on the terminology) Solomonoff Induction hooked up to an expectimax tree.

So it does have to examine every possible "universe" as a Turing Machine, but it uses proven-convergent probabilistic reasoning to get rid of the bad ones and raise good hypotheses to the top much faster than a universal search.

>The speed of intelligence scales. But not the quality of it. > >The quality of intelligence is something very hard to quantify. But as simply as I can state it, it's the quality of coming up with an idea that is not just an extension of an existing one, but an idea that came seemingly from nowhere. To someone else the idea appears magic, and no amount of thinking by that other person can lead him in a path that will end up there.

To be frank, before I'll believe this I want to see studies done on either humans or a mathematical examination of formal models of intelligent agents.

I think "never" is much too strong a word, considering that we seem to have a huge abundance of people capable of learning calculus even though few can rediscover it from scratch. There appear to exist very many people who would require such great efforts to learn certain things that doing so would be unpractical, but very few who simply cannot learn things no matter how much effort they put in.