News for hackers. Stuff that matters.
Re:The Last Invention of Man by thaumaturgy on Friday October 06, 2017 @06:10PM (#15420837)
Yeah. Maciej wrote a pretty good piece rebutting AI alarmism and kind of alludes to that as one of several points.

http://idlewords.com/talks/superintelligence.htm

> With no way to define intelligence (except just pointing to ourselves), we don't even know if it's a quantity that can be maximized. For all we know, human-level intelligence could be a tradeoff. Maybe any entity significantly smarter than a human being would be crippled by existential despair, or spend all its time in Buddha-like contemplation.

and

> But the hard takeoff scenario requires that there be a feature of the AI algorithm that can be repeatedly optimized to make the AI better at self-improvement.

And so on. It's a good read.

Re:The Last Invention of Man by apsec112 on Friday October 06, 2017 @07:37PM (#15421265)
Rebuttal to that essay by the Machine Intelligence Research Institute (AI safety group):

https://intelligence.org/2017/01/13/response-to-ceglowski-on...

Re:The Last Invention of Man by crooked-v on Friday October 06, 2017 @08:00PM (#15421360)
That write-up seems to mainly try to shrug off the "what is intelligence" issue by virtue of "maximization", while ignoring that when dealing with human intelligence, selecting for maximization of a single field gets you autistic savants, not Lex Luthor-grade manipulative geniuses.
Re:The Last Invention of Man by SiVal on Friday October 06, 2017 @09:15PM (#15421663)
My own rebuttal to that unimpressive Maciej argument is that he merely listed many ways in which something that seemed as though it ought to work didn't work out at all as expected, so we can safely assume that something as challenging as "superintelligence" won't work out either, and is therefore no danger.

But the argument that there are countless ways for plans to go awry and instead cause something unexpected to happen is hardly reassuring evidence that there is, therefore, "nothing to worry about".

Re:The Last Invention of Man by spynxic on Friday October 06, 2017 @07:27PM (#15421213)
> With no way to define intelligence (except just pointing to ourselves..

..and anything else considered to be alive)

Not to make an attempt at arguing whether or not machines can be intelligent but machines are not submerged in an environment that effects them, giving them no merit to be intelligent about anything; and even if they were they still have no ability to influence the environment. (Sensors for example)

It seems many that speak on AI have completely disregarded coming up with a reasonable implementation of intelligence in favor of pondering its potential uses. (Typically in ideals central to humans)

Re:The Last Invention of Man by xarball on Sunday October 08, 2017 @02:21AM (#15427187)
> machines are not submerged in an environment that effects them, giving them no merit to be intelligent about anything

You're alluding to the problem of having to 'raise' a super-intelligence with a vested interest in the continuity of its physical existence. I think I agree with where you're going with that.

Our understanding of 'self' or 'self preservation' likely comes from a long history of suffering, pain, and failure brought on by trials at understanding the self from evolution's attempts at imbuing the information for longevity into cognition. Mother nature has had years of exponential growth with an inconceivable number of branches attempting to understand the 'self', that lead to failure, all necessary because the idea of 'self preservation' or 'grounding one's sense of ego into a concept of physical sensory data' is very obscure to precisely define, let along to define such that it can be TESTED between generations.

Based on how much of a headstart evolution has on that refining process, and how bad humans are at implementing that or even exposing sensory information about that that plug directly into concepts that necessarily affect an artificial's sense of self, I'm willing to bet that we are completely ill-equipped to grow any real intelligence that's truly grounded in the real world, until it can make all the same failed attempts at living longer or evolving + testing those attempts, with that exact same directional goal of 'self-preservation' which humans have.

Self-preservation is really the only thing you can evolve to, to guide a physical evolution. I'm not aware of any other physical goals that could lead to the same human-like intelligence or human-like neural models.

Most creatures seem to hit the exact same point. An agent-based survival of the fittest with generational physical trials that punish poor understandings of the self's physical manifestation, trying to optimize for its longevity.

Anything that doesn't hit that kind of model is strictly not aware of the manifestation of its ego, as we know it (e.g. plants, bacteria, etc), and I don't think the physical sense of self can evolve into grounded concepts without allowing for generational failure on the understanding of self, due to incorrect concepts of 'physical self' between competing generations.

TL;DR - The concept of physical self is an evolved trait only if there is a goal tied to it, it cannot arrive without the concept of 'self existence' being threatened or when not allowed to fail.

Re:The Last Invention of Man by sampo on Friday October 06, 2017 @08:03PM (#15421381)
> Maybe any entity significantly smarter than a human being would be crippled by existential despair

This was addressed in a Spider-Man story where a villain enhances his intelligence, then first becomes a very successful crime boss, then writes novels and music, and then falls into existential despair.

https://en.wikipedia.org/wiki/Flowers_for_Rhino

Re:The Last Invention of Man by kobeya on Friday October 06, 2017 @08:19PM (#15421471)
I think it was done first, and probably better, in Flowers for Algernon.
Re:The Last Invention of Man by kwhitefoot on Saturday October 07, 2017 @03:16AM (#15422694)
Just in case anyone her is unfamiliar with the story here is a link to the Wikipedia synopsis: https://en.wikipedia.org/wiki/Flowers_for_Algernon.

Daniel Keyes should be more famous and the story more widely known.

And now for something weird: http://www.unboundworlds.com/2008/10/challenged-and-banned-f...

How anyone could think of banning one of the most affecting and humane books ever written is beyond me.

Re:The Last Invention of Man by perl4ever on Friday October 06, 2017 @09:35PM (#15421740)
"it is a pastiche of the science fiction story Flowers for Algernon."
Re:The Last Invention of Man by doktrin on Saturday October 07, 2017 @07:22AM (#15423346)
> pretty good piece rebutting AI alarmism

I only read the text, but I was under the impression it was mainly meant to be a funny Wat-style talk and not a meaningful rebuttal.

If, in fact, this piece is intended as a serious rebuttal then I honestly think it fails. The overwhelming majority of the content is either a funny whataboutism (e.g. Gilligan's Island, pot smoking roommate, the Great Emu War, etc.), or jokes about what could be loosely termed the "AI community".

AI 'alarmism', as I understand it, is not predicated on the idea that super-AI is a certainty - merely that it's a possibility. No point raised in this piece gives me a reason to believe that AGI (or whatever) is either impossible or so unlikely that it can be disregarded.

Re:The Last Invention of Man by saulrh on Friday October 06, 2017 @07:59PM (#15421356)
You're betting everything on vacuous what-ifs. I mean, let's turn this around:

With no way to define intelligence (except by pointing to ourselves), we don't even know if we're good at it. For all we know human intelligence is a dead end. Maybe any entity that is not a product of evolution, free of genetic detritus that becomes maladaptive around advanced technology, will outstrip human civilization like human civilization outstripped evolution.

I don't doubt the book is full of entertaining soundbytes, but given these two quotes I am unconvinced that it is any more than entertainment.

Re:The Last Invention of Man by westoncb on Friday October 06, 2017 @07:12PM (#15421132)
Similarly, I wonder if it's something where adding to it doesn't actually do much, like all the variations you see done on Turing Machines (adding more tapes or 2d tapes etc.) which seem like they would increase its capacity somehow, but it turns out they have the same power as the original Turing Machine. Or like the arithmetic of infinite numbers, e.g. aleph-0 + aleph-0 = aleph-0.
Re:The Last Invention of Man by felipemnoa on Friday October 06, 2017 @09:02PM (#15421622)
>>> With no way to define intelligence (except just pointing to ourselves), we don't even know if it's a quantity that can be maximized. For all we know, human-level intelligence could be a tradeoff. Maybe any entity significantly smarter than a human being would be crippled by existential despair, or spend all its time in Buddha-like contemplation.<<

The only intelligence that I believe is important is problem solving. If an AI is as capable as humans at problem solving then that is all you need.

>>crippled by existential despair

Sounds to me that you are talking about feelings. I really do not see the purpose of purposely giving the AI feelings. A strong AI should not have any feelings just like your google maps application does not have any feelings when giving you directions.

A general AI will simply be a general problem solver at the same level of a human but with a virtual infinite capacity to scale its computational abilities.

You could definitely give it free will so that it decides its own goals but why would anybody do something like that unless they want it to turn on them?

It would be like somebody launching a nuclear ballistic missile and letting it decide which city to land in. It is possible to program that type of free will but why? It could decide to destroy your city.

Re:The Last Invention of Man by perl4ever on Friday October 06, 2017 @09:30PM (#15421718)
Feelings regulate actions. People don't blindly follow rules that would lead to disaster because of more fundamental regulatory mechanisms such as feelings. In order to make an AI that is as flexible as a smart human, it can't be brittle, and therefore will need ways to break out of logical traps. That is what a general problem solver must be like, inherently.
Re:The Last Invention of Man by felipemnoa on Saturday October 07, 2017 @11:23AM (#15424099)
Feelings can be irrational so I really do not think you want to employ a mechanism that uses anything resembling feelings. If it can be irrational that means it can also go crazy and decide to exterminate all human kind.

>>In order to make an AI that is as flexible as a smart human, it can't be brittle, and therefore will need ways to break out of logical traps.

I think this can be achieved without having to give the machines flaws we have just like we did not need to create machines with flapping wings to fly.

Remember that the only goal of a living organism is to replicate and have its DNA live another generation. That's it. Feelings helps us achieve that goal I suppose. An AI does not need to have that goal. If anybody were to give it that goal it would only be a matter of time before it decided that humans were standing in the way of its goal.

Re:The Last Invention of Man by therein on Saturday October 07, 2017 @01:12AM (#15422374)
Very interesting point about using feelings and intuition to break out of logical traps. Does that also mean this AI will be fallible or perhaps even stubborn? Will it ever refuse to accept that it is wrong about a result it produces?
Re:The Last Invention of Man by CrystalGamma on Saturday October 07, 2017 @03:21PM (#15425247)
Where did you get that URL? It's not linked on http://idlewords.com/talks/ …
Re:The Last Invention of Man by 5_minutes on Friday October 06, 2017 @06:32PM (#15420953)
Thanks for this