Re:The Darwinian argument for worrying about AI
by ben_w on Friday June 30, 2023 @06:26PM
(#36543330)
> An undergrad can do this too.
Just to be clear:
You're asserting that an undergrad can produce a list of 40,000 deadly chemical agents, many novel, many involving no restricted components, via a process which also output VX nerve gas, overnight?
> To try to get the field regulated so they can build a moat, maybe?
To.
Not.
Die.
Why is this so hard to believe?
Especially in that specific field where the stuff was mostly already under control?
> Honestly, I'm just as shocked at your disposition here.
> We don't need laws to protect our feelings. The recent pushes to make everything into a safe space are making our culture dull and sterile. We're drawing lines around permissible thoughts and feelings and using censorship to control discourse.
> Human society should be robust, not coddled.
Can I infer you think this is about feelings?
It isn't.
A rude AI? Whatever, if that's the intent. Someone wants bad taste jokes, let them have it.
An AI which used threats, which tries to manipulate its users? Even if it's just a parrot, it's clearly not doing what it was supposed to do.
How is that not a warning sign?
Again, this isn't about coddling; at least, not beyond the fact that this is absolutely easy mode from the perspective of alignment, and we still can't manage it.
> Also, information isn't dangerous
Given what I just wrote, how is your imagination failing you so badly?
That specific information, how to make a dangerous substance, is obviously dangerous even if it was either of the two ways I know how to make chlorine gas within the specific constraints of the prompt — basic kitchen supplies.
> Scoped down, trusted computing, cordoned and firewalled off, meticulously studied and monitored, killed off on a regular interval, etc. etc. And that's probably before we'd see emergence of actual threatening capabilities.
You're literally not only asking for these to be open sourced, making it impossible, but also having a go at OpenAI for doing so much as half of that.
But at least we do agree that that list of things is an appropriate response, even if you are unwilling to see the risks in even current AI systems.
Just to be clear:
You're asserting that an undergrad can produce a list of 40,000 deadly chemical agents, many novel, many involving no restricted components, via a process which also output VX nerve gas, overnight?
> To try to get the field regulated so they can build a moat, maybe?
To.
Not.
Die.
Why is this so hard to believe?
Especially in that specific field where the stuff was mostly already under control?
> Honestly, I'm just as shocked at your disposition here.
> We don't need laws to protect our feelings. The recent pushes to make everything into a safe space are making our culture dull and sterile. We're drawing lines around permissible thoughts and feelings and using censorship to control discourse.
> Human society should be robust, not coddled.
Can I infer you think this is about feelings?
It isn't.
A rude AI? Whatever, if that's the intent. Someone wants bad taste jokes, let them have it.
An AI which used threats, which tries to manipulate its users? Even if it's just a parrot, it's clearly not doing what it was supposed to do.
How is that not a warning sign?
Again, this isn't about coddling; at least, not beyond the fact that this is absolutely easy mode from the perspective of alignment, and we still can't manage it.
> Also, information isn't dangerous
Given what I just wrote, how is your imagination failing you so badly?
That specific information, how to make a dangerous substance, is obviously dangerous even if it was either of the two ways I know how to make chlorine gas within the specific constraints of the prompt — basic kitchen supplies.
> Scoped down, trusted computing, cordoned and firewalled off, meticulously studied and monitored, killed off on a regular interval, etc. etc. And that's probably before we'd see emergence of actual threatening capabilities.
You're literally not only asking for these to be open sourced, making it impossible, but also having a go at OpenAI for doing so much as half of that.
But at least we do agree that that list of things is an appropriate response, even if you are unwilling to see the risks in even current AI systems.