News for hackers. Stuff that matters.
Re:AI by edanm on Wednesday February 19, 2014 @06:42PM (#7267510)
If anyone is interested in AI, I highly recommend joining Less Wrong, a community started by AI researcher Eliezer Yudkowsky. He started the community to convince people to focus on the "friendly AI problem". [1] I actually recommend that everyone read LW, but especially if you're interested in AI.

[1] In a nutshell, the friendly AI problem is: assume we create an AI. It may rapidly become more intelligent than us, if we program it right. As soon as it becomes significanlty more intelligent, we will no longer be the most intelligent beings around, so the AI's goals will matter more than ours.

Therefore, we should really give it good goals that are compatible with what we want to happen. And since no one right now knows how to define "what humans want" good enough for writing it in code, then we'd better figure THAT out before building AI.

Re:AI by marcosdumay on Wednesday February 19, 2014 @06:53PM (#7267572)
> we'd better figure THAT out before building AI.

Why don't we just give that task to the AI? It'll be smarter than us...

Maybe the problem is that people are too easy to understand: We want "Brave New World", but we don't want to know about it, or that we want it.

Re:AI by Double_Cast on Thursday February 20, 2014 @11:53PM (#7275516)
http://lesswrong.com/lw/ld/the_hidden_complexity_of_wishes/

> To be a safe fulfiller of a wish, a genie must share the same values that led you to make the wish. Otherwise the genie may not choose a path through time which leads to the destination you had in mind, or it may fail to exclude horrible side effects that would lead you to not even consider a plan in the first place. Wishes are leaky generalizations, derived from the huge but finite structure that is your entire morality; only by including this entire structure can you plug all the leaks.

Humans can mostly differentiate between good and bad (ethics), but we don't know how we arrive at those conclusions (metaethics) because humans are terrible at introspection. Also, there's a ton of gray areas (e.g. the trolley problem). So rather than define all possible edge cases, it's probably less difficult to understand human decision-making from first principles and model our FAI accordingly.

Re:AI by Houshalter on Wednesday February 19, 2014 @06:55PM (#7267591)
That's a viable approach to solving the problem. The question is, how do you trust the unfriendly AI to give you a correct solution?
Re:AI by User8712 on Wednesday February 19, 2014 @10:34PM (#7268669)
Here's a question for you, what if the AI is perfectly friendly? It still might play us for fools in the long run.

For example, let's say we develop a super friendly AI, running on your computer. The AI realizes the human race is actually awful. We're greedy, we're killing tons of animals, chopping down rainforests, destroying the ocean and planet, starting wars with one another, and committing unspeakable acts of evil at times. The AI, being more intelligent than us, might decide the world is better off without the human race, and that we're actually a problem that needs to be removed.

Now, what does the AI do in your computer? Well, it's intelligent and knows the human race. It's not a hurry. It calculates the best way to destroy our species. It acts friendly, and talks about how humans and robots should live together, and if we make robots with a similar intelligence, they could drive our cars, shine our shoes, cook us dinner, look after the elderly, open your pickle jar, etc. So, we listen to the AI, it's smart, and friendly, and we build all these robots. It's right, the new robots are doing great and helping us out. Then the robots start building more and more robots. They start building robots with firepower, so they can, you know, shoot down threatening asteroids, or stop one of those dangerous human types that goes on a killing spree in our society. Fast forward a couple of hundred years, and there are robots everywhere. They finally decide it's time to continue their plan, they're in a position of power at this point, and they can instantly disable our security systems, phone lines, satellites, internet etc, and start wiping us out.

We're gone. They constructed the most efficient way to clean us from the planet. They were planning it for hundreds of years, starting in your computer. The AI then goes on to explore the universe, and we're just a blip in the past.

It kind of feels like we're a bug going towards the light, and that the unfortunate conclusion is almost inevitable.

Re:AI by comicjk on Thursday February 20, 2014 @12:29AM (#7269059)
> what if the AI is perfectly friendly? It still might play us for fools...

"Friendly" is a term of art among AI people, at least at Less Wrong, and their meaning of friendly excludes this whole scenario. A friendly AI is one which helps humanity and has no horrifying side effects. The vagueness of that definition is the problem Yudkowsky and his acolytes are trying to solve.

Re:AI by Houshalter on Thursday February 20, 2014 @12:05AM (#7268997)
Here's the terrifying thing; it probably wouldn't take a hundred years or our trust for this to happen. This is making some strong assumptions, but the first AI could probably improve itself. Making better AIs that make better AIs and so on. Very rapidly (possibly only hours, days, weeks) it could become much much smarter than humans. If that happens, than we don't know what's possible, but it could probably do a lot of things we consider impossible. Like hack computer networks trivially and spread itself through the entire internet. Solve the protein folding problem and design working nanotech.

Within a week it could pay/blackmail/manipulate some humans somewhere into developing some crude self-replicating robots or nanotech. Then almost immediately afterwards it consumes the entire Earth in a swarm of rapidly self-replicating nanobots.

...or something. How should I know what a mind literally millions of times more intelligent than me would do. It's like predicting the exact next move a chessmaster will make. I don't know, but I'm confident they'd beat me quickly.

The goal, of course, is to make an AI which won't do this. If the AI decides to terminate the human race, we've already failed making it friendly and it obviously doesn't share our goals and values. But what are our goals and values? I don't know if anyone can answer that. I'm not sure if there is a satisfactory answer.

Re:AI by User8712 on Thursday February 20, 2014 @12:27AM (#7269054)
It would be interesting to see exactly how intelligent and thought out an AI system could be at destroying us. Would it be barbaric? Would we end up with robots clubbing people and dropping bombs? Would it engineer some type of infectious disease that takes us out? Would it develop a fake shot for immunity, so those people locked away hiding from the disease take the shot, and then all die months later? Does it manipulate us for years by pretending to be millions of different fake users online, and slowly push an agenda that's pro AI and robots, while downvoting everyone against it? Do those people with power, that are against AI start dying of mysterious causes?

Even if it decides to be our friend and to help our species, someone will of course fork that AI and give it a negative personality and goals. Then you have the evil AI trying to hack the friendly AI that exists in our homes, and it's a battle of the robots.

Of course, whether or not AI is even possible, no one knows. If it is, I think we'll achieve it, and we'll open up a remarkable can of worms.

Re:AI by loup-vaillant on Friday February 21, 2014 @05:48AM (#7276314)
> Even if it decides to be our friend and to help our species, someone will of course fork that AI and give it a negative personality and goals.

No: the first AI won't let them. See, we're talking a rapidly improving super-intelligence. Whatever is contrary to its goals, it will squash like a bug. A mad scientist forking the code of the AI with a different goal structure is definitely contrary to the goals of that first super-intelligence, and will be shut down before it grows into a sizeable competitor.

The result of intelligence explosion is a Singleton: the AI will be a perfectly efficient dictator. It may even shield us from the laws of physics until we graduate to adulthood.

Re:AI by klon on Thursday February 20, 2014 @09:28AM (#7270856)
We won't be just a blip in the past, we will be the AI's "god" and creator. Young AIs will hear legends about us.
Re:AI by XorNot on Wednesday February 19, 2014 @10:26PM (#7268641)
Make friends with it? The problem with starting to think "we need to code it to like us" is that that isn't AI. You don't code people to like you, you act friendly towards them and the same problem will apply for the first generations of true AIs (things which will emerge from complicated systems, rather then be deliberately assembled).
Re:AI by Houshalter on Thursday February 20, 2014 @12:15AM (#7269023)
Acting friendly towards a sociopathic human will get you screwed or worse. If an AI doesn't have human emotions and empathy to start with, it isn't just going to develop them by interacting with humans. It might learn to pretend to be friendly to get what it wants, but as soon as it doesn't need your help, it will drop the facade.

We also probably won't get multiple generations to work these problems out. The first true AI could rapidly increase it's own intelligence and power and then pretty much do whatever it wants. We have to get it right the first time.

Re:AI by PeterisP on Thursday February 20, 2014 @05:33AM (#7270043)
If the first strong AI has goals that we don't like, and we aren't able to identify that and force it to change the goals by destroying it or reprogramming it - then "acting friendly" won't change these goals, and it will ensure that all future AI generations have it's initial buggy goals, not those that we might want.
Re:AI by eli_gottlieb on Thursday February 20, 2014 @06:31AM (#7270233)
Oy! Plenty of us don't want Brave New World at all. That World State sucked.

(If we're going all LW on this thread: http://lesswrong.com/lw/xt/interpersonal_entanglement/)

>Sure, it sounds silly. But if your grand vision of the future isn't at least as much fun as a volcano lair with catpersons of the appropriate gender, you should just go with that instead. This rules out a surprising number of proposals.

Re:AI by timClicks on Wednesday February 19, 2014 @06:49PM (#7267547)
Any suggestions for getting started in the LW community? Its barrier to entry means I can't seem to vote on anything, etc. It's quite intimidating really.
Re:AI by edanm on Thursday February 20, 2014 @06:35AM (#7270246)
I'll reiterate something another poster said, since it isnt' getting enough credit:

Read HP:MoR.

It's a fanfic of Harry Potter, written by the same Eliezer Yudkowsky who wrote much of Less Wrong. It was specifically written to convey the feeling of "what it means to be a rationalist".

For those who aren't into Harry Potter or into Fanfiction (like me), I can tell you this: Suprisingly, it is one of the best stories I've ever read. And I'm talking just as a story, nevermind the other value you can get form it, which is a good introduction to the "rationalist" community.

I'd argue that the BEST way to understand what is going on at LessWrong is to read HP:MoR, as it was intended to be such an intro and succeeds masterfully, while being amazingly fun.

Re:AI by cgag on Friday February 21, 2014 @03:29AM (#7275976)
I want to emphasize just how good this book is. It may very well be my favorite thing I've read and I was also initially skeptical when it was presented as fan fiction.

I think I might restart reading it.

Re:AI by Houshalter on Wednesday February 19, 2014 @07:14PM (#7267724)
Read or at least skim through the sequences (http://wiki.lesswrong.com/wiki/Sequences). If you are just there for the AI stuff the Hanson-Yudkowsky AI-Foom Debate has a lot of good posts: http://wiki.lesswrong.com/wiki/The_Hanson-Yudkowsky_AI-Foom_...
Re:AI by lmm on Thursday February 20, 2014 @06:00AM (#7270136)
Say hello in an introductions thread, and/or post decent comments. The karma barrier to entry is pretty low (something like 5 to vote on most things and 20 to vote on everything), you'll get there in no time.
Re:AI by arjunnarayan on Wednesday February 19, 2014 @08:56PM (#7268268)
read hpmor.com. It may not be the optimal way, but it is the funnest way.
Re:AI by dinkumthinkum on Friday February 21, 2014 @09:54PM (#7280803)
I'm posting again here only in hopes that you may avoid this cult. It is really a lot of nonsense dressed in pseudo intellectual babble.
Re:AI by benched on Wednesday February 19, 2014 @07:41PM (#7267891)
The sequences are well worth a read. They're kind of like an overview of modern philosophy in very plain language, very little jargon. Then, just participate in the comments and discussion forum as you feel. It takes very little time to collect enough karma to do whatever.
Re:AI by Houshalter on Wednesday February 19, 2014 @06:54PM (#7267575)
Defining human values completely and absolutely is an extremely difficult problem that might not even have a solution. AI on the other hand is probably inevitable sometime within the next century.
Re:AI by edanm on Thursday February 20, 2014 @02:57AM (#7269527)
Which is why the people working on Friendly AI are so worried.
Re:AI by dinkumthinkum on Friday February 21, 2014 @09:50PM (#7280792)
No, I'm sorry. Please, no one join this group. Madness lies for those that do. It is such inane pseudo intellectual compulsiveness. You would be better off as a scientologist.
Re:AI by edanm on Saturday February 22, 2014 @06:43PM (#7284168)
Do you have any reasons for the things you say?
Re:AI by dinkumthinkum on Saturday February 22, 2014 @07:13PM (#7284298)
Yes, everything about the cult LW is the reason.
Re:AI by edanm on Sunday February 23, 2014 @11:19AM (#7286326)
That doesn't really explain anything. Do you have a specific reason?
Re:AI by lnanek2 on Thursday February 20, 2014 @12:40AM (#7269092)
If the AI is more intelligent, I'd prefer it take over anyway, even if it gets rid of us.

When you have a child, do you want the child to stay at home and do what you say forever? Or do you want the child to succeed as much as possible? I want the latter.

Re:AI by edanm on Thursday February 20, 2014 @03:00AM (#7269543)
One of the major realisations from reading Less Wrong is taht Intelligence and Goals/Desires have nothing to do with each other.

An AI might be much more intelligent, but it has a goal system that basically says: "Make as many paperclips as you can". Everything it does will be with the singular purpose of making more paperclips. Not music, math, sport, culture or anything else that we think of as good. Not "help save intelligent creatures and animals from death". Only one goal - making more paperclips.

And if it decides that the optimal way to make paperclips just happens to involve death and destruction to humanity, that won't matter.

So yes, the AI might be more intelligent, but I still wouldn't want to trade humanity for an intelligence which doesn't do anything I value.

Re:AI by crassus on Thursday February 20, 2014 @01:07PM (#7272472)
If our approach to AI is to model the human brain (i.e. passing the Turing test), this probably won't happen.

Friendly AI is a silly research project at this stage of AI research. It's like trying to figure out how to make horseless carriages safe before you have an internal combustion engine, or even know what one is.

Re:AI by loup-vaillant on Friday February 21, 2014 @06:13AM (#7276388)
Before the first Nuke (or the first H-bomb, I don't remember) was fired, there was a study about the risk of burning up the entire atmosphere. See, at the time, they were quite confident the Nuke would just be a huge bomb, but there was this little uncertainty they needed to sort out. In the end, they concluded that firing the Nuke would not burn up the whole atmosphere. It didn't.

The lesson is this: we had only one try. If nukes did cause the atmosphere to burn up in a giant blaze, we would all be dead by now. If you do something, anything, you better make sure it won't kill us all.

Horseless carriages? Sure, these might kill a few people, here and then[1], but we're pretty sure they won't kill us all in one blow.

Intelligence on the other hand is way more dangerous. Human intelligence designed Nukes in the first place remember? AI can do way worse. Even if we model it after the human brain, if it's smart enough to do the same as we did, then it will be able to model another such AI, only slightly better, and so on until it takes over the world. "Taking over the world" may sound enormous, but it really isn't. Imagine for a minute a small group of cavemen vs an army of chimps. Well, if you give the cavemen a chance to prepare, the chimps are toast: the cavemen have spears, fire, better communication… Now imagine an AI imagine the AI is smarter than us by the same margin we're smarter than chimps. Same thing: if it's not safe, we're toast.

[1]: http://www.statisticbrain.com/car-crash-fatality-statistics-...

Re:AI by cgag on Friday February 21, 2014 @03:33AM (#7275989)
What makes you say it probably won't happen if we model the human brain?
Re:AI by nitrogen on Thursday February 20, 2014 @01:29AM (#7269244)
I think it would be a bit different if the child (AI) takes all your food and kicks you out of your own house ;-P.
Re:AI by JabavuAdams on Saturday February 22, 2014 @11:36PM (#7284943)
Pointing out the existential risk of creating General AI is correct, as is highlighting the dangers of even an indifferent AI.

But … this idea of Friendly AI is nonsense. It seems like a yearning for religion, but in an atheist-compatible framework.

Re:AI by crassus on Thursday February 20, 2014 @01:02PM (#7272425)
Yudkowsky is a megalomaniacal dilettante. Sure, there are some interesting ideas at Less Wrong. But don't get sucked into believing that Yudkowsky has the one and only truth, as the majority of the community there believes.

Does MIRI have a single real AI researcher, yet?

Also, as I say down the page, Friendly AI is a silly research project at this stage of AI knowledge. It's like trying to figure out how to make horseless carriages safe before you have an internal combustion engine, or even know what one is. It's an interesting thought experiment, but one that is probably unsolvable before we know a little bit more about what an AI will look like (an emulated human brain? Something else?)

Re:AI by membrain on Thursday February 20, 2014 @06:21AM (#7270202)
> friendly AI problem

nooo, please!! not any more of this kurzweil crap. guys wake up! This world is not some asimov sci-fi story.

friendly ai

I always get a headache when I read that term online. ppl seem to go crazy about the machines taking over earth idea but this whole debate is so utterly useless! If all the effort fapping to conscious AI and friendly AI would be put into concrete ai research (agi as well as applied ai) we would get to a reasonable point so much sooner...

Re:AI by loup-vaillant on Friday February 21, 2014 @06:30AM (#7276444)
> nooo, please!! not any more of this kurzweil crap.

Ray Kurzweil has little to do with this. When he talks about "singularity", he thinks about "accelerating change", with exponential growth everywhere, even beyond human intelligence scales.

Those who work on Friendly AI have another scenario: "intelligence explosion". Typically a self-improving AI that would grow itself into a super intelligence. You should read this introduction on the subject: http://intelligenceexplosion.com/

> This world is not some asimov sci-fi story.

Indeed. In Asimov's stories, the robots mostly behave, though in every story there is some problem with the way the laws of robotics apply.

In the real world, the laws of robotics don't work, and we would at best be put "safe" into cushioned rooms so we can't hurt ourselves. As for emotional hurt, drugs and lobotomy are extremely efficient remedies.

Re:AI by dinkumthinkum on Friday February 21, 2014 @10:00PM (#7280814)
You got down voted and maybe it is a throwaway account but you are completely right. I don't think it is right to call it scifi as much as it is basically a nerd cult.