News for hackers. Stuff that matters.
Re:Eliezer Yudkowsky on large language model economics by jerry1979 on Tuesday March 14, 2023 @10:30AM (#35152104)
What do you mean by "L. Ron Hubbard's style guide"? Would that be doing things like employing in-group metaphors?
Re:Eliezer Yudkowsky on large language model economics by PaulHoule on Tuesday March 14, 2023 @11:17AM (#35152876)
Making up lots of vocabulary that divides insiders from outsiders.

I was interested in Scientology as a microcosm of society around the time alt.religion.scientology appeared in the early 1990s. Scientology documented everything about how it runs, even management is scriptural, and only Hubbard could write scripture so it is like a specimen that's been fixed to a microscope slide.

Since then renegade "Free Zone" Scientologists have put all of his writings online so it is very accessible.

Dianetics is the Sequences of Scientology in that it is a long, windy, and nonsensical book that selects for people who are perseverant readers who are impervious to critical thinking. (Like one of those Nigerian scam emails.)

Hubbard's hypnotic language is demonstrated in his Philadelphia Doctorate Lectures where he not only mentions the hypnotist Milton Erickson but uses Erickson's techniques extensively. (There is no record of Hubbard meeting Erickson but Hubbard and Erickson were both highly active at the same time in Arizona.)

(That is, that weird ramblely kind of writing is by no means innocent but it is misparsed by many people's minds and helps create an altered state of consciousness.)

Hubbard's most satanic (in opposition to conventional Eastern & Western religious ideas) works are Introduction to Scientology Ethics and A New Slant on Life. The first one describes an elaborate system of punishments that are applied to anyone who violates group norms and a system of evaluation that guarantees you will violate those norms by accident. The list of "ethics conditions" is folded so the positive conditions come first and a person who starts reading at the beginning and doesn't go all the way through will miss the batshit crazy stuff. The second describes how a Scientologist is supposed to disconnect themselves from anyone who disagrees with their spiritual journey.

Something Scientology shares with the EY cult (as well as the LaRouche organization and the third international) is the proliferation of front groups. Scientology has Narconon, Criminon, The Way to Happiness Foundation, and many others. The EY cult has "effective altruism", "rationalism", "longtermism", "AI safety" and probably those parties Aella used to run.

It's a typical path that a coercive group makes multiple 360° turns. "Effective Altruism" seems designed to attract morally weak people but seems rational enough at first. I mean, it seemed to make sense when Bill Gates insisted on funding only international NGOs that could prove the value of what they were doing.

Once, however, you are funding "longtermist" organizations that are concerned with imagined problems on the way between here and the glorious future that we've turned all the planetary mass of the Milky Way into Dyson spheres there are no results to measure!

It is like the technique a stage hypnotist uses of putting volunteers through a number of screens that rapidly selects the most suggestible and/or likely to go along subjects. Both in the short term and long term this process expels the trouble makers and leave behind the potential Manchurian Candidates.

Re:Eliezer Yudkowsky on large language model economics by krrrh on Tuesday March 14, 2023 @11:34AM (#35153194)
As another former a.r.s. and a.c.t. denizen this is spot on. Good summary!
Re:Eliezer Yudkowsky on large language model economics by Kinrany on Tuesday March 14, 2023 @11:31AM (#35153122)
You're mixing bad things that Hubbard does and Eliezer doesn't with things that are done by both but are not obviously bad.

It reads like a "Hitler also ate sugar" argument.

Re:Eliezer Yudkowsky on large language model economics by PaulHoule on Tuesday March 14, 2023 @11:48AM (#35153446)
I bring up Hubbard because Hubbard documented his organization so well that he's a good target. There is a long list of other organizations that I could also find analogies with.

It’s true Hubbard divided the world into ‘public’ who would have their bank accounts empty and ‘staff’ who, penniless, would be turned into worker drones.

EY is not building that sort of machine. But Hubbard was operating at peak egalitarianism in the US and EY is operating at a time (and places) that have peak inequality. A Scientology whale gives a few million to the organization, the ideal EY whale is Sam Bankman Fried. Since it is money that is fungible and ‘makes the world go round’ he doesn’t need an army of worked drones when just one rich kid at Stanford or Cambridge can be quietly bled.

Other groups to look at with apocalyptic ideology are the Symbionese Liberation Army (Why make worker drones when you can brainwash Patty Hearst?), People's Temple (...go to San Francisco), Heaven’s Gate, Aum Shinrikyo, etc. It amazes me that one of them hasn’t tried to assassinate an A.I. researcher yet.

Re:Eliezer Yudkowsky on large language model economics by bondarchuk on Tuesday March 14, 2023 @01:20PM (#35154957)
This is not a response to Kinrany's argument, in fact your last line is doing exactly what you're accused of again.

>Other groups to look at with apocalyptic ideology are the Symbionese Liberation Army (Why make worker drones when you can brainwash Patty Hearst?), People's Temple (...go to San Francisco), Heaven’s Gate, Aum Shinrikyo, etc. It amazes me that one of them hasn’t tried to assassinate an A.I. researcher yet.

Not obviously bad: have apocalyptic ideology (might actually be a little bit bad, admittedly)

Obviously bad but Eliezer et al. haven't done it: try to assassinate a researcher

Re:Eliezer Yudkowsky on large language model economics by the_af on Tuesday March 14, 2023 @12:17PM (#35153900)
Thanks for writing this.

It seems EY is now less fashionable, so criticizing him as a crackpot is more acceptable. But I remember a few years ago when he was in full swing, he had a lot of traction even here on HN, and of course in many nerdy circles. Also see that debacle with Roko's Basilisk and all those silly antics. The LessWrong crowd is insufferable.

I can sort of understand why EY had so much pull with internet nerds. He talked the talk, he was confident, and he was a nerd. But I'm also relieved that more people seem to understand now that he is a crackpot, and that claiming one is an expert on AI doesn't make it so, and that writing long essays mixing Bayes, Harry Potter and "rationalism" doesn't really make anyone an expert on anything.

Re:Eliezer Yudkowsky on large language model economics by bondarchuk on Tuesday March 14, 2023 @01:12PM (#35154795)
Do you have any argument besides "he writes long-windedly with insular terminology" and "there are multiple organizations affiliated to him"? E.g.

>It's a typical path that a coercive group makes multiple 360° turns.

What is the evidence that Eliezer is running a coercive group? Whom are they coercing into what?

I fully get that LessWrong et al. are obnoxious, use their own terminology, have far-out ideas about the future of AI that are probably wrong, weird moral systems (though any moral system is weird when you look at it hard enough imho)... But you are making the charge that Eliezer is running a cult and doing so on purpose, which is more specific than just running an annoying community around some non-mainstream ideas.

Re:Eliezer Yudkowsky on large language model economics by AnimalMuppet on Tuesday March 14, 2023 @01:22PM (#35155001)
Hmm. Reading this, I'm seeing a similar vibe with Urbit.