News for hackers. Stuff that matters.
Re:Language models can explain neurons in language models by ftxbro on Tuesday May 09, 2023 @01:23PM (#35877528)
> "This work is part of the third pillar of our approach to alignment research: we want to automate the alignment research work itself."

I feel like this isn't a Yud-approved approach to AI alignment.

Re:Language models can explain neurons in language models by FeepingCreature on Tuesday May 09, 2023 @01:30PM (#35877637)
Honestly, I think any foundational work on the topic is inherently Yud-favored, compared to the blithe optimism and surface-level analysis at best that is usually applied to the topic.

Ie, I think it's not that this shouldn't be done. This should certainly be done. It's just that so many more things than it should be done before we move forward.

Re:Language models can explain neurons in language models by snapcaster on Tuesday May 09, 2023 @01:37PM (#35877755)
Agreed, Yud does seem to have been right about the course things will take but I'm not confident he actually has any solutions to the problem to offer
Re:Language models can explain neurons in language models by qumpis on Tuesday May 09, 2023 @01:39PM (#35877800)
Which things has he been right about and when, if you recall?
Re:Language models can explain neurons in language models by snapcaster on Wednesday May 10, 2023 @01:23PM (#35890464)
At least to me, this current generation of AI progress is looking a lot more "foomy" than a lot of people besides him predicted
Re:Language models can explain neurons in language models by causalmodels on Tuesday May 09, 2023 @02:11PM (#35878253)
His solution is a global regulatory regime to ban new large training runs. The tools required to accomplish this are, IMO, out of the question but I will give Yud credit for being honest about them while others who share his viewpoint try to hide the ball.
Re:Language models can explain neurons in language models by snapcaster on Thursday May 11, 2023 @09:17AM (#35901592)
Good point, but I think if he was serious he would be looking to Serena Butler for inspiration not nuclear non-proliferation treaties
Re:Language models can explain neurons in language models by Teever on Tuesday May 09, 2023 @01:35PM (#35877724)
Why does this matter?
Re:Language models can explain neurons in language models by shadowgovt on Tuesday May 09, 2023 @01:26PM (#35877595)
"Yud-approved?"
Re:Language models can explain neurons in language models by ftxbro on Tuesday May 09, 2023 @01:32PM (#35877670)
He's the one in the fedora who is losing patience that otherwise smart sounding people are seriously considering letting AI police itself https://www.youtube.com/watch?v=41SUp-TRVlg
Re:Language models can explain neurons in language models by jack_riminton on Tuesday May 09, 2023 @01:53PM (#35877992)
That's not a fedora, that's his King of the Redditors crown
Re:Language models can explain neurons in language models by aitanabewa on Tuesday May 09, 2023 @01:32PM (#35877679)
Re:Language models can explain neurons in language models by circuit10 on Tuesday May 09, 2023 @01:30PM (#35877636)
Re:Language models can explain neurons in language models by bick_nyers on Tuesday May 09, 2023 @01:42PM (#35877851)
These were my thoughts exactly. On one hand, this can enable alignment research to catch up faster. On the other hand, if we are worried about homicidal AI, then putting it in charge of policing itself (and training it to find exploits in a way) is probably not ideal.
Re:Language models can explain neurons in language models by ShamelessC on Tuesday May 09, 2023 @02:27PM (#35878472)
You mean Yudkowski? I saw him on Lex Friedman and he was entirely unconvincing. Why is everyone deferring to a bunch of effective altruism advocates when it comes to AI safety?
Re:Language models can explain neurons in language models by ftxbro on Tuesday May 09, 2023 @02:54PM (#35878819)
> Why is everyone deferring to a bunch of effective altruism advocates when it comes to AI safety?

I'm not sure Yudkowski is an EA, but the EAs want him in their polycule.

Re:Language models can explain neurons in language models by tomjakubowski on Tuesday May 09, 2023 @03:10PM (#35879021)
He posts on the forum. I'm not sure what more evidence is needed that he's part of it.

https://forum.effectivealtruism.org/users/eliezeryudkowsky

Re:Language models can explain neurons in language models by ftxbro on Tuesday May 09, 2023 @03:23PM (#35879167)
I guess it's true, not just a rationalist but also effective altruist!
Re:Language models can explain neurons in language models by lubesGordi on Tuesday May 09, 2023 @03:24PM (#35879190)
I heard him on Lex too, and it seemed to be just a given that AI is going to be deceptive and want to kill us all. I don't think there was a single example of how that could be accomplished given. I'm open to hearing thoughts on this, maybe I'm not creative enough to see the 'obvious' ways this could happen.
Re:Language models can explain neurons in language models by ethanbond on Tuesday May 09, 2023 @03:26PM (#35879222)
This is also why I go into chess matches against 1400 elo players. I cannot conceive of the specific ways in which they will beat me (a 600 elo player), so I have good reason to suspect that I can win.

I'm willing to bet the future of our species on my consistent victory in these types of matches, in fact.

Re:Language models can explain neurons in language models by lubesGordi on Tuesday May 09, 2023 @03:50PM (#35879522)
Again, a given that AI is adversarial. Edit: In addition, as an 1100 elo chess player, I can very easily tell you how a 1600 player is going to beat me. The analogy doesn't hold. I'm in good faith asking how AI could destroy humanity. It seems given the confidence people who are scared of AI have in this, that they have some concrete examples in mind.
Re:Language models can explain neurons in language models by ethanbond on Tuesday May 09, 2023 @03:53PM (#35879550)
No it’s a given that some people who attempt to wield AI will be adversarial.

In any case a similar argument can be made with merely instrumental goals causing harm: “I am an ant and I do not see how or why a human would cause me harm, therefore I am not in danger.”

Re:Language models can explain neurons in language models by lubesGordi on Tuesday May 09, 2023 @03:59PM (#35879637)
People wielding AI and destroying humanity is very different from AI itself, being a weird alien intelligence, destroying humanity.

Honestly if you have no examples you can't really blame people for not being scared. I have no reason to think this ant-human relationship is analogous.

And seriously, I've made no claims that AI is benign so please stop characterizing my claims thusly. The question is simple, give me a single hypothetical example of how an AI will destroy humanity?

Re:Language models can explain neurons in language models by ethanbond on Tuesday May 09, 2023 @04:05PM (#35879693)
Sure, here’s a trivial example: It radicalizes or otherwise deceives an employee at a virus research lab into producing and releasing a horrific virus.

The guy at Google already demonstrated that AIs are able to convince people of fairly radical beliefs (and we have proof that even humans a thousand years ago were capable of creating belief systems that cause people to blow themselves up and kill thousands of innocent people).

P.S. I was not characterizing your opinion, I was speaking in the voice of an ant.

Re:Language models can explain neurons in language models by FeepingCreature on Wednesday May 10, 2023 @02:10AM (#35884209)
If AI is general, it will also have the capability of wielding itself, and probably better than humans.
Re:Language models can explain neurons in language models by kevinventullo on Tuesday May 09, 2023 @03:58PM (#35879616)
Other caveman use fire to cook food. Fire scary and hurt. No understand fire. Fire cavemen bad.
Re:Language models can explain neurons in language models by ethanbond on Tuesday May 09, 2023 @04:06PM (#35879703)
Other caveman use nuke to wipe out city. Nuke scary and hurt. No understand nuke. Nuke caveman bad.

Other caveman use anthrax in subway station. Anthrax scary and hurt…

Is AI closer to fire or closer to nukes and engineered viruses? Has fire ever invented a new weapon system?

By the way: we have shitloads of regulations and safety systems around fire due to, you guessed it, the amount of harm it can do by accident.

Re:Language models can explain neurons in language models by PeterisP on Tuesday May 09, 2023 @03:57PM (#35879610)
IMHO the argument isn't that AI is definitely going to be deceptive and want to kill us all, but rather that if you're 90% sure that AI is going to be just fine, that 10% of existential risk is simply not acceptable, so you should assume that this level of certainty isn't enough and you should act as if AI may be deceptive and may kill us all and take very serious preventive measures even if you're quite certain that it won't be needed - because "quite certain" isn't enough, you want to be at "this is definitely established to not lead to Skynet" level.
Re:Language models can explain neurons in language models by NumberWangMan on Tuesday May 09, 2023 @08:51PM (#35882307)
And even with all that, probably it's best to still exercise an abundance of caution, because you might have made a mistake somewhere.
Re:Language models can explain neurons in language models by ethanbond on Tuesday May 09, 2023 @03:24PM (#35879196)
Because they have arguments that AI optimists are unable to convincingly address.

Take this blog post for example, which between the lines reads: we don't expect to be able to align these systems ourselves, so instead we're hoping these systems are able to align each other.

Consider me not-very-soothed.

FWIW, there are plenty of AI experts who have been raising alarms as well. Hinton and Christiano, for example.

Re:Language models can explain neurons in language models by ryan93 on Tuesday May 09, 2023 @03:34PM (#35879326)
People won't care until an actually scary AI exists. Will be easy to stop at that point. Or you can just stop research here and hope another country doesn't get one first. Im personally skeptical it will exist. Honestly might be making it worse with the scaremongering coming from uncharismatic AI alignment people.
Re:Language models can explain neurons in language models by ethanbond on Tuesday May 09, 2023 @03:43PM (#35879442)
Why would it be easy to stop at that point? The believable value prop will increase in lockstep with the believable scare factor, not to mention the (already significant) proliferation out of ultra expensive research orgs into open source repos.

Nuclear weapons proliferated explicitly because they proved their scariness.

Re:Language models can explain neurons in language models by ryan93 on Tuesday May 09, 2023 @04:03PM (#35879671)
If AI can exist humans have to figure it out. It’s what we do. Really shockingly delusional to think people are gonna use chatgpt for a few min get bored and then ban it like it’s a nuke. I’d rather the USA get it first anyways.
Re:Language models can explain neurons in language models by ethanbond on Tuesday May 09, 2023 @04:19PM (#35879865)
Where did I say we could or should ban it like a nuke?

Anyway this is a good example of the completely blind-faith reasoning that backs AI optimism: we’ll figure it out “because it’s what we do.”

FWIW we have still not figured out how to dramatically reduce nuclear risk. We’re here just living with it every single day still, and with AI we’re likely stepping onto another tightrope that we and all future generations have to walk flawlessly.

Re:Language models can explain neurons in language models by sebzim4500 on Tuesday May 09, 2023 @08:02PM (#35881952)
>If AI can exist humans have to figure it out. It’s what we do.

We have figured out stuff in the past, but we also came shockingly close to nuclear armageddon more than once.

I'm not sure I want to roll the dice again.

Re:Language models can explain neurons in language models by Kostchei on Tuesday May 09, 2023 @11:45PM (#35883406)
Are you kidding? "easy to stop"? When LLMs are integrated into law enforcement, banking, research, education, logistics... All these areas have people building backend systems leveraging the current llm tech and pipelines to plug in the coming tech. If we reach a tipping point where these things become aware/act truly autonomously, what are the chances they do it before we notice? People are renowned for implementing things before understanding the consequences.

And what does charisma of AI alignment folks have to do with anything?

Re:Language models can explain neurons in language models by killthebuddha on Tuesday May 09, 2023 @03:06PM (#35878965)
DISCLAIMER: I think Yudkowsky is a serious thinker and his ideas should be taken seriously, regardless of whether not they are correct.

Your comment triggered a random thought: A perfect name for Yudkowsky et al and the AGI doomers is... wait for it... the Yuddites :)

Re:Language models can explain neurons in language models by throwaway2137 on Tuesday May 09, 2023 @04:50PM (#35880201)
Already used on 4chan :)