The hype chamber in SV is on overdrive on AI at this point.
Singularity AI concept has gone from the neat Sci Fi idea it was, to being treated as a serious possibility. This is Absurd. The modern version of Malthus, and a community of people who pride themselves on their reason, should do a basic sniff test.
The author has it correct - human brains themselves are significantly impressive on a weight/power/capability scale.
But forget all of the intelligence debate, consider the things that people magically ignore: emotions.
Take a look at how people who are depressed function. They measured as being more aware than (better assessment of their surroundings/relations/self assesment) people who were not.
At the same time, they lack motivation, or desire - there is a lack of impulsive force to move forward.
Intelligence doesn't give human beings purpose. Emotion does.
This isn't feel good psychobabble, its a fundamental part of good mind/body housekeeping which a huge chunk of SV follows every day to reach "peak performance".
How are you going to create a general purpose AI which has any motive force?
Its actually MORE LIKELY, that if a GP AI is created, its just going to sit on its ass and pretend not to understand instructions.
So would an explicit goal function, which is how we already give "purpose" to algorithms.
> How are you going to create a general purpose AI which has any motive force?
In any other out of countless of ways to do that. What makes you think that emotions are necessary to create a powerful optimization process?
The "motive force" is mostly a solved problem for now, we can code that explicitly (the issue is with figuring out a right goal and how to write it down). AI development is mostly about the way it works, not why it should do the work.
--
No offense meant to you personally, but I find that most of the comments about "overdriven AI dangers focus in SV" to reveal that authors don't have a fucking clue what the issue is about, and never spent time actually reading up the reasoning behind the AI X-risk potential.
I'll give an ultra-compressed super-TLDR of that reasoning for benefit of future conversations. It goes like this:
- intelligence is a super-strong optimization process; it doesn't necessarily have to look the way humans think (humans generally suck at reasoning, which is well established; see: cognitive biases, probability theory, decision theory, etc.)
- intelligence is an independent factor from values/goals, a mind can have any combination of the two - i.e. just because it's smart, it doesn't mean it will develop the same morality humans do, or any morality whatsoever; see https://wiki.lesswrong.com/wiki/Orthogonality_thesis
- combining the two, the danger of super-human AI is not something that's hostile to us - it's something that's indifferent about us, and is more powerful than us, the same way we don't give a second thought about e.g. ants
That's interesting. Do you have a reference for that?
This is me if i was an AI.
1. Bootstrap. In order to really take off, AI needs to be able to build better AI.
2. AI needs to be adaptable. Any human go player can play on 19x20 board. AlphaGo (a program that that on 19x19 board beats top professionals 60:0) could not even make a legal move.
3. AI should be resilient. Right now it runs on hardware made by humans and blindly relies on that hardware working perfectly all the time.
You can think of intelligence as a couple of dimensions rather than just one dimension as the author is arguing. However I don't agree with his/her logic.
You can think of the brain as a massively parallel computer. Intelligence can be gauged by total cores, memory and CPU per core, power usage and network bandwidth between cores. Sure different configurations will give different strengths but something optimized atomically for performance will be faster than a mushy brain.
Now imagine a billion of this little pods placed all over the planet. With amazing bandwidth between the computers akin to our internet.
Worst case: they think humans are the enemy and out think us to extinction.
Best case: studying intelligence helps us figure out how Brians work. This helps us figure out how DNA nano manufacturing works. We figure out brain to brain and brain to AI networking. Now all humans are collectively a super computer being able to share knowledge and abilities. May be this is extended so animals are also included and Earth becomes a Pandora like planet where every living thing is connected and they all live in harmonius resource optimized world where AI tries to maximize happiness of every living being.