News for hackers. Stuff that matters.
Re:Eliezer Yudkowsky on the Singularity, Bayesian Brains and... by retbull on Thursday March 10, 2016 @12:14PM (#11260438)
For all that I loved his hpmor book I can't stand to read him speaking the way he does. It feels like he tries too hard to sound smart and has built his own hill that he gets to be king of and ignore the rest of the world from the top of.
Re:Eliezer Yudkowsky on the Singularity, Bayesian Brains and... by dsjoerg on Thursday March 10, 2016 @12:31PM (#11260539)
I disagree that he tries too hard to sound smart. I despise the tendency to unnecessarily use complex words when simple ones will do. Police officers, for example, are rightly famous for this tendency. In Eliezer's writing and in this Q&A, the complex words are there because they are the most concise and precise expression of what he's trying to say -- simpler words would either be inaccurate or would have to go on at length.

For example:

Horgan: Is college overrated?

Yudkowsky: It'd be very surprising if college were underrated, given the social desirability bias of endorsing college.

You'd need a few paragraphs to explain what Yudkowsky means to people who don't already understand social desirability bias.

Re:Eliezer Yudkowsky on the Singularity, Bayesian Brains and... by snowwrestler on Thursday March 10, 2016 @01:22PM (#11260931)
Yudkowsky has a way of thinking that he believes everyone else should adopt. He is skilled at using phrasing to frame his statements in his way of thinking.

For example, instead of saying something like, "a lot of people think college is good," he phrases it in terms of a bias. Why? Because if he can get the other person talking about the subject in terms of biases, then he has advanced his own favorite way of thinking.

But is liking college really a bias? And if so, compared to what norm? He offers no explanation; he just asserts it.

Re:Eliezer Yudkowsky on the Singularity, Bayesian Brains and... by greenrd on Sunday March 13, 2016 @11:56AM (#11278094)
I don't think he's saying that liking college is a bias. Or rather, he is, but I think he's saying something subtly different (which implies the former): that those who denigrate college, or are "meh" about college, take a "social desirability hit" - they are likely to be seen as either unintelligent manual labourers, or Philistines, or annoying self-taught outliers, or insulting people who have put in the hard work to go to college, or a combination. And therefore young people grow up in such an environment and many of them have a strong belief, in such an environment, that college is a necessity, and don't really consider the evidence dispassionately.
Re:Eliezer Yudkowsky on the Singularity, Bayesian Brains and... by rayalez on Thursday March 10, 2016 @01:44PM (#11261137)
- Is college overrated?

- Yes. People keep endorsing college education because it makes them seem smart.

Re:Eliezer Yudkowsky on the Singularity, Bayesian Brains and... by gizmo on Thursday March 10, 2016 @02:20PM (#11261452)
Except that is a totally different explanation (and much less plausible).

Example: Parents are not motivated by seeming smart nor do they get credit for acting smart when they push their kids to go to college because it's the default position. Sending your kids to college is simply what "respectable people" do. This plausibly leads to a social desirability bias.

I'm pretty skeptical about the magnitude of this effect, though. I would say the popularity of higher education can be better explained by the zero-sum nature -- and hence arms race -- of competition for jobs.

Re:Eliezer Yudkowsky on the Singularity, Bayesian Brains and... by retbull on Thursday March 10, 2016 @01:13PM (#11260849)
I actually really liked that line because it was beautifully concise I was talking more about the essay answers later in the interview.
Re:Eliezer Yudkowsky on the Singularity, Bayesian Brains and... by saint_fiasco on Thursday March 10, 2016 @01:24PM (#11260948)
I think those are Yudkowsky desperately trying to explain things without using scientific technical terms and lesswrongian jargon.

You can see he kind of drops character with the 'cyborg' question.

Re:Eliezer Yudkowsky on the Singularity, Bayesian Brains and... by StavrosK on Friday March 11, 2016 @09:43PM (#11271077)
> Even if some of the wilder speculations are true and it's possible for our universe to spawn baby universes, that doesn't get us literal immortality. To live significantly past a googolplex years without repeating yourself, you need computing structures containing more than a googol elements, and those won't fit inside a single Hubble volume.

> And a googolplex is hardly infinity. To paraphrase Martin Gardner, Graham's Number is still relatively small because most finite numbers are very much larger. Look up the fast-growing hierarchy if you really want to have your mind blown, well, eternity is longer than that. Only weird and frankly terrifying anthropic theories would let you live long enough to gaze, perhaps knowingly and perhaps not, upon the halting of the longest-running halting Turing machine with 100 states.

This literally says "eternity is very very long".

Re:Eliezer Yudkowsky on the Singularity, Bayesian Brains and... by greenrd on Sunday March 13, 2016 @11:58AM (#11278102)
Yes, this was by far the most excruciating part of the interview.
Re:Eliezer Yudkowsky on the Singularity, Bayesian Brains and... by Moshe_Silnorin on Thursday March 10, 2016 @12:27PM (#11260512)
Yudkowsky writes without modesty and doesn't mind seeming arrogant. This doesn't particularly bug me; in fact I kind of like it - I think he's a fascinating guy. But a lot of people find his writing style intolerable, and he has a disproportionate number of antifans.
Re:Eliezer Yudkowsky on the Singularity, Bayesian Brains and... by gaze on Thursday March 10, 2016 @02:19PM (#11261440)
Enemies. Adversaries? Anti-fans? Really?
Re:Eliezer Yudkowsky on the Singularity, Bayesian Brains and... by Moshe_Silnorin on Thursday March 10, 2016 @04:33PM (#11262556)
They aren't enemies in the same way a fan isn't a friend. I would say "antifan" describes a distinct thing.
Re:Eliezer Yudkowsky on the Singularity, Bayesian Brains and... by rjeli on Thursday March 10, 2016 @05:15PM (#11262907)
...critic?
Re:Eliezer Yudkowsky on the Singularity, Bayesian Brains and... by p1esk on Friday March 11, 2016 @05:43PM (#11270084)
Actually, "antifan" is the right word here. "Don't care/know enough to criticize", and "don't hate enough to be an enemy", but simply think he's overrated.
Re:Eliezer Yudkowsky on the Singularity, Bayesian Brains and... by rdbcasillas on Thursday March 10, 2016 @12:24PM (#11260493)
Well, if you have read most of his lesswrong articles(like I have), its hard to not enjoy the way he speaks and writes about these things. But I totally get that it feels obtuse and unnecessarily complex to someone without good exposure to his http://lesswrong.com/ writings.
Re:Eliezer Yudkowsky on the Singularity, Bayesian Brains and... by andrewprock on Thursday March 10, 2016 @12:31PM (#11260534)
I think that's a big part of the disconnect. Unless you are a devotee of EY, you generally don't "get" him. I suspect that it's because he mostly operates in the echo chamber that is the "less wrong" community.

In short, he generally does not get credit for what he knows and understands because he cannot communicate clearly with people who do not already know and understand his philosophy.

Said another way, if you can't explain things to people unless they already understand you, then you can't explain things to people.

Re:Eliezer Yudkowsky on the Singularity, Bayesian Brains and... by TheOtherHobbes on Thursday March 10, 2016 @02:45PM (#11261693)
I don't get him at all. To me he looks like a Randian dilettante folk philosopher, with a minimal track record of practical achievement.

His AI arguments seem to be of the "If you assume an AGI singularity is possible..." kind, which is suspiciously like assuming god exists, for completely arbitrary values of god, and then debating whether or not he cares enough to hate you.

Most software today sucks so badly it can't tie its own shoelaces without poking itself in the brain and falling over.

How do we know this isn't a general feature of software? Has there ever been a 100% bullet-proof non-trivial software system? Is there anything in the history of CS which implies that such a system is possible?

So suggesting that by (say) 2066 software will not only stop sucking but will be able to meta-improve itself with bug-free perfection to the point where it can eat the world seems curiously ungrounded.

Re:Eliezer Yudkowsky on the Singularity, Bayesian Brains and... by snowwrestler on Thursday March 10, 2016 @04:49PM (#11262687)
You're right about the quality. But in addition AI, as we build it today, is task-oriented, not personality-oriented. People are trying to build cars that drives themselves where directed, not cars that decide whether or not they feel like driving people around at all.

Now, maybe it's possible that a task-oriented AI will at some point develop a personality. But there is no evidence that such a thing is possible.

The "real life" personalities we observe in nature--ourselves--did not evolve from task-oriented machinery. It was the opposite: personality and will is observable in even simple animals; a beetle directs its own actions and tries above all else to preserve its own life. No one sets a task for it.

Where does a beetle get that self-direction? Well, what is life itself? Where did it come from? We find ourselves in very deep waters; the depths of which I think AI-fearing folks don't seem to appreciate.

Re:Eliezer Yudkowsky on the Singularity, Bayesian Brains and... by greenrd on Sunday March 13, 2016 @12:13PM (#11278155)
I don't think it is the case that "AI-fearing folks" like myself don't appreciate what the alleged difficulties of building AGI might be. Rather, the burden of proof is on you, the doubters, to show that we need to spend no money at all on such risks, despite the magnitude of the harm that would be caused if we're right and nothing is done about the risks.
Re:Eliezer Yudkowsky on the Singularity, Bayesian Brains and... by greenrd on Sunday March 13, 2016 @12:07PM (#11278132)
> So suggesting that by (say) 2066 software will not only stop sucking but will be able to meta-improve itself with bug-free perfection to the point where it can eat the world seems curiously ungrounded.

That's a straw man. Software doesn't need to be bug-free to take over the world, literally or figuratively. Cars replaced horses but weren't fault-free. PCs replaced typewriters but weren't bug-free. All software needs to do to take over the world is be sufficiently competent and superior to its competitors - products or (in the case he's concerned about) humans.

Also, it's true that one AI might blow itself up, or make a fatal mistake, while trying to take over the world, but it only needs one idiot misconfiguring his AI in his basement to give us a potential world-destroying calamity - the classic "paperclip-maximiser" problem. That's a very hard problem, and it's bigger than solving the AI control problem - it's no good knowing the secret to keeping AIs under control with probability 0.9999999999999999... if someone else unleashes a dangerous AI without knowing that secret, or deliberately ignores it due to insanity or some death-wish on humanity.

And while he doesn't focus on this, there are plenty of AI risks that we still might conceivably face even if we defeat the outright existential risks, the threats to our very existence as a species. For example, consider what would happen if a paedophile or a rapist got hold of a very powerful AI and was able to command it...

Re:Eliezer Yudkowsky on the Singularity, Bayesian Brains and... by noobiemcfoob on Thursday March 10, 2016 @01:12PM (#11260838)
Yes, yes and more yes. I love his ideas and thoughts on many topics, but it always comes across as the person who was too busy being "brilliant" to learn how to interface well with the broader society. The sad part is that he really is brilliant.
Re:Eliezer Yudkowsky on the Singularity, Bayesian Brains and... by armitron on Thursday March 10, 2016 @02:01PM (#11261291)
No he's not, he's just skilled at manipulating people's perceptions of him based on a narrow set of premises that he introduced.

Go through his non-philosophical papers and point to one thing in there that's brilliant. His miniscule technical output also seems to agree with this.

I weep for the world that considers people like Yudkowsky brilliant.

Re:Eliezer Yudkowsky on the Singularity, Bayesian Brains and... by noobiemcfoob on Friday March 11, 2016 @10:43AM (#11267220)
There are many definitions for "brilliant", though most of them heavily involve philosophy from my perspective.

The easiest example of his work that I'd describe as brilliant is HPMOR. It is a story that makes the Harry Potter world not just bearable but beautiful and lively. There are portions where the philosophy comes on so thick as to disrupt the story, but on the whole it is a beautiful work. Reading it is how I first discovered EY.

Re:Eliezer Yudkowsky on the Singularity, Bayesian Brains and... by p1esk on Friday March 11, 2016 @05:49PM (#11270111)
I haven't read HPMOR, and I haven't read Harry Potter books, but I watched the movies, and I think HP world is beautiful as is. What makes it unbearable for you?
Re:Eliezer Yudkowsky on the Singularity, Bayesian Brains and... by rdbcasillas on Thursday March 10, 2016 @02:11PM (#11261370)
Point taken. But there are some complex ideas that just need some amount of effort from the reader in order to "get" them. I don't know what is the right balance but if you are being introduced to new ideas, introduction of those ideas will also bring some vocabulary that you will (at least initially) be uncomfortable with. I guess I want to see someone communicate the ideas on less wrong in a simpler way without dumbing down or ignoring the nuances.
Re:Eliezer Yudkowsky on the Singularity, Bayesian Brains and... by greenrd on Sunday March 13, 2016 @12:15PM (#11278162)
I'm amused at the irony of users on a site devoted to technology, many of whom are probably software engineers, complaining about obscure jargon...
Re:Eliezer Yudkowsky on the Singularity, Bayesian Brains and... by throw12122 on Thursday March 10, 2016 @12:33PM (#11260549)
Is this a cult? I enjoy the was Halmos and Feynman talk about things. I've never read anything interesting from Yudkowsky...
Re:Eliezer Yudkowsky on the Singularity, Bayesian Brains and... by Moshe_Silnorin on Thursday March 10, 2016 @12:43PM (#11260603)
I don't think it was a cult. LessWrong was an interesting place where interesting people gathered for a time; though LessWrong is basically on life support now.

Gwern, Hal Finny, Wei Dai, Hannu Rajaniemi, Aaron Swartz, and Scott Alexander are some of the more well-known (outside of LessWrong) former members. It really was an interesting community for a while. There were meet-ups and Yudkowsky did solicit for his nonprofit, but it never seemed culty to me. And none of the people I mentioned ever seemed wary of criticizing Yudkowsky when they thought he was wrong. See, for example, Gwern's epic reply to Yudkowsky's post on automation: http://lesswrong.com/lw/hh4/the_robots_ai_and_unemployment_a...

This doesn't seem like the type of reply an entranced follower would give their guru. The fact that this very-critical comment got twice as many upvotes as the original post is telling of the attitude of the community as a whole.

Re:Eliezer Yudkowsky on the Singularity, Bayesian Brains and... by greenrd on Sunday March 13, 2016 @12:18PM (#11278181)
Worth noting that the community has spread out into a kind of "diaspora": some have gone into effective altruism (many retaining their AI risk beliefs, especially the ones in California); some have been hired to work on existential risks full-time; Scott Alexander now has his own (impressive) blog and some LWers still follow him there and go to meetups that he attends... Of course, it's also probably true that some have since graduated, got jobs and simply have less time to read and post stuff on the internet any more.
Re:Eliezer Yudkowsky on the Singularity, Bayesian Brains and... by dTal on Thursday March 10, 2016 @03:13PM (#11261923)
Depends what you mean by "cult". They are a defined group of people that display a substantial amount of in-group closeness (google "lesswrong cuddle puddle" if you have the stomach for it), possessing unusual views, organised around a central leader figure, that believes the leader's worldview is superior. Poisoned kool-aid, however, is unlikely. Make of it what you will.
Re:Eliezer Yudkowsky on the Singularity, Bayesian Brains and... by retbull on Thursday March 10, 2016 @01:11PM (#11260826)
I've read a lot of his work, but I don't think it needs to be as complex as it is. I am smart, well educated, and well read and I often have to pull out a dictionary to read what he writes. Maybe I just need to read more but I don't feel like he is trying to communicate his ideas only listen to himself speak.
Re:Eliezer Yudkowsky on the Singularity, Bayesian Brains and... by leeleelee on Thursday March 10, 2016 @03:01PM (#11261844)
When I read this, I actually came away with the impression that he was able to accurately and eloquently express certain ideas/beliefs/concepts that I agree with and can grasp in my own mind, but I would be unable to express these same ideas coherently to another human being.