News for hackers. Stuff that matters.
Re:Eliezer Yudkowsky on the Singularity, Bayesian Brains and... by rdbcasillas on Thursday March 10, 2016 @12:24PM (#11260493)
Well, if you have read most of his lesswrong articles(like I have), its hard to not enjoy the way he speaks and writes about these things. But I totally get that it feels obtuse and unnecessarily complex to someone without good exposure to his http://lesswrong.com/ writings.
Re:Eliezer Yudkowsky on the Singularity, Bayesian Brains and... by andrewprock on Thursday March 10, 2016 @12:31PM (#11260534)
I think that's a big part of the disconnect. Unless you are a devotee of EY, you generally don't "get" him. I suspect that it's because he mostly operates in the echo chamber that is the "less wrong" community.

In short, he generally does not get credit for what he knows and understands because he cannot communicate clearly with people who do not already know and understand his philosophy.

Said another way, if you can't explain things to people unless they already understand you, then you can't explain things to people.

Re:Eliezer Yudkowsky on the Singularity, Bayesian Brains and... by TheOtherHobbes on Thursday March 10, 2016 @02:45PM (#11261693)
I don't get him at all. To me he looks like a Randian dilettante folk philosopher, with a minimal track record of practical achievement.

His AI arguments seem to be of the "If you assume an AGI singularity is possible..." kind, which is suspiciously like assuming god exists, for completely arbitrary values of god, and then debating whether or not he cares enough to hate you.

Most software today sucks so badly it can't tie its own shoelaces without poking itself in the brain and falling over.

How do we know this isn't a general feature of software? Has there ever been a 100% bullet-proof non-trivial software system? Is there anything in the history of CS which implies that such a system is possible?

So suggesting that by (say) 2066 software will not only stop sucking but will be able to meta-improve itself with bug-free perfection to the point where it can eat the world seems curiously ungrounded.

Re:Eliezer Yudkowsky on the Singularity, Bayesian Brains and... by snowwrestler on Thursday March 10, 2016 @04:49PM (#11262687)
You're right about the quality. But in addition AI, as we build it today, is task-oriented, not personality-oriented. People are trying to build cars that drives themselves where directed, not cars that decide whether or not they feel like driving people around at all.

Now, maybe it's possible that a task-oriented AI will at some point develop a personality. But there is no evidence that such a thing is possible.

The "real life" personalities we observe in nature--ourselves--did not evolve from task-oriented machinery. It was the opposite: personality and will is observable in even simple animals; a beetle directs its own actions and tries above all else to preserve its own life. No one sets a task for it.

Where does a beetle get that self-direction? Well, what is life itself? Where did it come from? We find ourselves in very deep waters; the depths of which I think AI-fearing folks don't seem to appreciate.

Re:Eliezer Yudkowsky on the Singularity, Bayesian Brains and... by greenrd on Sunday March 13, 2016 @12:13PM (#11278155)
I don't think it is the case that "AI-fearing folks" like myself don't appreciate what the alleged difficulties of building AGI might be. Rather, the burden of proof is on you, the doubters, to show that we need to spend no money at all on such risks, despite the magnitude of the harm that would be caused if we're right and nothing is done about the risks.
Re:Eliezer Yudkowsky on the Singularity, Bayesian Brains and... by greenrd on Sunday March 13, 2016 @12:07PM (#11278132)
> So suggesting that by (say) 2066 software will not only stop sucking but will be able to meta-improve itself with bug-free perfection to the point where it can eat the world seems curiously ungrounded.

That's a straw man. Software doesn't need to be bug-free to take over the world, literally or figuratively. Cars replaced horses but weren't fault-free. PCs replaced typewriters but weren't bug-free. All software needs to do to take over the world is be sufficiently competent and superior to its competitors - products or (in the case he's concerned about) humans.

Also, it's true that one AI might blow itself up, or make a fatal mistake, while trying to take over the world, but it only needs one idiot misconfiguring his AI in his basement to give us a potential world-destroying calamity - the classic "paperclip-maximiser" problem. That's a very hard problem, and it's bigger than solving the AI control problem - it's no good knowing the secret to keeping AIs under control with probability 0.9999999999999999... if someone else unleashes a dangerous AI without knowing that secret, or deliberately ignores it due to insanity or some death-wish on humanity.

And while he doesn't focus on this, there are plenty of AI risks that we still might conceivably face even if we defeat the outright existential risks, the threats to our very existence as a species. For example, consider what would happen if a paedophile or a rapist got hold of a very powerful AI and was able to command it...

Re:Eliezer Yudkowsky on the Singularity, Bayesian Brains and... by noobiemcfoob on Thursday March 10, 2016 @01:12PM (#11260838)
Yes, yes and more yes. I love his ideas and thoughts on many topics, but it always comes across as the person who was too busy being "brilliant" to learn how to interface well with the broader society. The sad part is that he really is brilliant.
Re:Eliezer Yudkowsky on the Singularity, Bayesian Brains and... by armitron on Thursday March 10, 2016 @02:01PM (#11261291)
No he's not, he's just skilled at manipulating people's perceptions of him based on a narrow set of premises that he introduced.

Go through his non-philosophical papers and point to one thing in there that's brilliant. His miniscule technical output also seems to agree with this.

I weep for the world that considers people like Yudkowsky brilliant.

Re:Eliezer Yudkowsky on the Singularity, Bayesian Brains and... by noobiemcfoob on Friday March 11, 2016 @10:43AM (#11267220)
There are many definitions for "brilliant", though most of them heavily involve philosophy from my perspective.

The easiest example of his work that I'd describe as brilliant is HPMOR. It is a story that makes the Harry Potter world not just bearable but beautiful and lively. There are portions where the philosophy comes on so thick as to disrupt the story, but on the whole it is a beautiful work. Reading it is how I first discovered EY.

Re:Eliezer Yudkowsky on the Singularity, Bayesian Brains and... by p1esk on Friday March 11, 2016 @05:49PM (#11270111)
I haven't read HPMOR, and I haven't read Harry Potter books, but I watched the movies, and I think HP world is beautiful as is. What makes it unbearable for you?
Re:Eliezer Yudkowsky on the Singularity, Bayesian Brains and... by rdbcasillas on Thursday March 10, 2016 @02:11PM (#11261370)
Point taken. But there are some complex ideas that just need some amount of effort from the reader in order to "get" them. I don't know what is the right balance but if you are being introduced to new ideas, introduction of those ideas will also bring some vocabulary that you will (at least initially) be uncomfortable with. I guess I want to see someone communicate the ideas on less wrong in a simpler way without dumbing down or ignoring the nuances.
Re:Eliezer Yudkowsky on the Singularity, Bayesian Brains and... by greenrd on Sunday March 13, 2016 @12:15PM (#11278162)
I'm amused at the irony of users on a site devoted to technology, many of whom are probably software engineers, complaining about obscure jargon...
Re:Eliezer Yudkowsky on the Singularity, Bayesian Brains and... by throw12122 on Thursday March 10, 2016 @12:33PM (#11260549)
Is this a cult? I enjoy the was Halmos and Feynman talk about things. I've never read anything interesting from Yudkowsky...
Re:Eliezer Yudkowsky on the Singularity, Bayesian Brains and... by Moshe_Silnorin on Thursday March 10, 2016 @12:43PM (#11260603)
I don't think it was a cult. LessWrong was an interesting place where interesting people gathered for a time; though LessWrong is basically on life support now.

Gwern, Hal Finny, Wei Dai, Hannu Rajaniemi, Aaron Swartz, and Scott Alexander are some of the more well-known (outside of LessWrong) former members. It really was an interesting community for a while. There were meet-ups and Yudkowsky did solicit for his nonprofit, but it never seemed culty to me. And none of the people I mentioned ever seemed wary of criticizing Yudkowsky when they thought he was wrong. See, for example, Gwern's epic reply to Yudkowsky's post on automation: http://lesswrong.com/lw/hh4/the_robots_ai_and_unemployment_a...

This doesn't seem like the type of reply an entranced follower would give their guru. The fact that this very-critical comment got twice as many upvotes as the original post is telling of the attitude of the community as a whole.

Re:Eliezer Yudkowsky on the Singularity, Bayesian Brains and... by greenrd on Sunday March 13, 2016 @12:18PM (#11278181)
Worth noting that the community has spread out into a kind of "diaspora": some have gone into effective altruism (many retaining their AI risk beliefs, especially the ones in California); some have been hired to work on existential risks full-time; Scott Alexander now has his own (impressive) blog and some LWers still follow him there and go to meetups that he attends... Of course, it's also probably true that some have since graduated, got jobs and simply have less time to read and post stuff on the internet any more.
Re:Eliezer Yudkowsky on the Singularity, Bayesian Brains and... by dTal on Thursday March 10, 2016 @03:13PM (#11261923)
Depends what you mean by "cult". They are a defined group of people that display a substantial amount of in-group closeness (google "lesswrong cuddle puddle" if you have the stomach for it), possessing unusual views, organised around a central leader figure, that believes the leader's worldview is superior. Poisoned kool-aid, however, is unlikely. Make of it what you will.
Re:Eliezer Yudkowsky on the Singularity, Bayesian Brains and... by retbull on Thursday March 10, 2016 @01:11PM (#11260826)
I've read a lot of his work, but I don't think it needs to be as complex as it is. I am smart, well educated, and well read and I often have to pull out a dictionary to read what he writes. Maybe I just need to read more but I don't feel like he is trying to communicate his ideas only listen to himself speak.