News for hackers. Stuff that matters.
Re:The Doomsday Invention by skybrian on Sunday December 13, 2015 @04:44PM (#10728096)
I'm wondering how much Eliezer Yudkowsky and the LessWrong crowd were influenced by Bostrom or vice versa. This all sounds very familiar, but the dates seem earlier:

"Bostrom introduced the philosophical concept of 'existential risk' in 2002."

Re:The Doomsday Invention by Animats on Sunday December 13, 2015 @06:03PM (#10728361)
"Bostrom introduced the philosophical concept of 'existential risk' in 2002."

Oh, come on. This has been a theme in science fiction back to at least the 1940s. Jack Williamson's "With Folded Hands" (1947) is probably the clearest early writing on the subject.

"The car turned off the shining avenue, taking him back to the quiet splendor of his prison. His futile hands clenched and relaxed again, folded on his knees. There was nothing left to do."

Re:The Doomsday Invention by j1o1h1n on Sunday December 13, 2015 @07:36PM (#10728676)
Also Frederic Brown, "Etaoin Shrdlu", 1942, which I have only read synopses of.
Re:The Doomsday Invention by MichaelMoser123 on Sunday December 13, 2015 @09:36PM (#10729118)
Also Karel Čapek wrote R.U.R in 1920; (well its about assembled artificial biological beings, so you can say that it is not about AI).

https://en.wikipedia.org/wiki/R.U.R.

And there is the Golem, which is even earlier.

Also i find it funny how much attention this theoretical problem receives, when we don't have a good explanations on what intelligence is and how it works.

Re:The Doomsday Invention by MichaelMoser123 on Monday December 14, 2015 @01:58PM (#10733012)
Also all these stories seem to be projecting our own behavior on robots; Humans and apes like to be alpha males, because it gives them an evolutionary advantage; by extension it seems natural that Robots will also have the same drive for power; is that really true?

I think that the Robot will not have such an in built desire - if it is based on reasoning then the machine might actually be more reasonable. In other words there is enough room for both Humans and Robots.

Re:The Doomsday Invention by MichaelMoser123 on Tuesday December 15, 2015 @01:00PM (#10739273)
Mr. Yudkowsky says: "Moore's Law of Mad Science: Every eighteen months, the minimum IQ necessary to destroy the world drops by one point." http://www.azquotes.com/quote/819025

The point is that our gadgets become more powerful and more potent as weapons; so as we have a good history of past abuses it is sort of easy to extrapolate to the future.

What is still don't understand: would intelligence be an inhibiting factor - like more reasonable Humans are supposedly less destructive; Maybe the same goes for machines.

I don't buy the argument about competition for resources - With enough effort you can always stretch it so that there is enough for everybody/everything.

Another interesting aspect: once upon a time people would become very agitated when discussing politics (that was when we still had ideological differences and when people thought that their stance does matter); In our time we have discussions about sci-fi instead of that.

Re:The Doomsday Invention by Animats on Sunday December 13, 2015 @08:20PM (#10728830)
"It believes what it sets", about a sentient Linotype.

That's a very funny short story. A magic story, though, not AI.

Re:The Doomsday Invention by paulsutter on Sunday December 13, 2015 @06:12PM (#10728389)
How about 1863?

"The first registered thoughts on the possibility of artificial, machine intelligence, becoming a risk to humanity stems from the late Industrial Revolution. It was Samuel Butler, in 1863, in his Darwin Among the Machines that suggested machines could eventually replace humans as the dominant agents on Earth"

https://wiki.lesswrong.com/wiki/History_of_AI_risk_thought

Re:The Doomsday Invention by eurg on Sunday December 13, 2015 @05:24PM (#10728242)
Bostrom brought up the orthogonality idea before Eliezer, who consequently (not immediately) changed his mind.

EDIT: I am unable to find the source. Might have been a halluzination, but I thought I remembered them having a discussion where Eliezer more or less still thought "let the AGI win, humanity be damned". Ah, my brain and I are not in a good relationship.

Re:The Doomsday Invention by skybrian on Sunday December 13, 2015 @07:56PM (#10728746)
Although he clearly didn't come up with the idea (there's the whole nuclear war thing, after all), perhaps it wasn't called "existential risk" before? Google ngram viewer show hits going back to the 1930's [1], but judging by the brief snippets, they seem to mostly be related to religion.

[1] https://books.google.com/ngrams/graph?content=existential+ri...