I agree. Overall the whole situation feels like we’ve just entered atomic age and are proliferating plutonium, while selling shiny radioactive toys [I’m actually pretty serious here, the effects of prolonged interactions with an AI haven’t been evaluated yet, technically there is even a possibility of overriding a weak personality].

But it still feels like it is much safer to let GPT-4 loose and assess the consequences. If compared to developing GPT-8 in private and letting it leak accidentally.

Yes. The only possible way this gets taken seriously is if a mediocre AI tries a power move, causes some damage, and faceplants before it's too big to stop.

I would not be surprised, if GPT-4 (in its optimal environment, with access to a well-working external memory, prompted in a right way, etc) is already capable enough to do an interesting power move.

For sure. Just to be clear, I'm not saying the situation we're in where we have to release it to the general public is a great situation to be in. But I think we're at a point where there's not any optimal solutions, only tradeoffs.

Alright who plugged gpt into his hn account?

> technically there is even a possibility of overriding a weak personality

Similar reflections here. There was even a site called GPT My Life that lets you delegate planning your day to GPT. I imagine this is a proto-version of that.

My opinion is that the "self-appointed betters" scenario is the lesser of two evils - it is still evil but there is no going back on that now.

As to "white hat researchers who have their own AI bot swarm", the assumption here is that the swarm can be controlled like some sort of pet. Since even at this early stage no one has a clue how GPT (say) actually manages to be as clever as it is, the assumption is not warranted when looking into the future.

Given that GPT-4 can already write image prompts as well or better than humans can, it wouldn't be surprising if it could convince any other AI to join it and override the white hats running the "private swarm".

So there's this little problem, say we can track X number of malware with the current security defenders. But ChatGPT lowers the barrier to programmer to a point which the average teenage could do malware, we now get a wave of hacks going on.. like whats the response there?

Have the AI working to plug the holes before the script kiddies can exploit them?

Such an obvious solution that any silly monkey can think it up.

And who trains it to find the problems? now your building scanning tools and if you've lucky you can get it to print out a commit to fix it if you have access to the code..

> And who trains it to find the problems?

I think maybe, pure speculation on my part, at least one out of the thousands of employees at Microsoft has been given the job to train their $10 billion investment on the windows code base to see what they can do about beefing up security.

And every other hour there’s some startup advertising this in the form of “Show HN”.

At this point I think people are just looking for reasons to fear the eventual AI extinction event without even trying.

The discussion needs to go beyond the model.

We need to talk about the training set for GPT and the process around RLHF.

Yes, the training data comes from people, and people are corrupt, illogical, random, emotional, unmotivated, take shortcuts, cheat, lie, steal, invent new things, and lead boring lives as well as not so boring lives. Expect the same behaviors to be emulated by a LLM. Garbage in = garbage out is still true with AI.

And the predominant mode of thought at OpenAI is that alignment can be achieved though RL, but we also know that this doesn’t actually work because you can still jailbreak the model. Yet they are still trying to build ever stronger egg shells. However much you RLHF the model to pretend to be nice, it still has all of the flawed human characteristics you mention on the inside.

RLHF is almost the worst thing you can do to a model if your goal is safety. Better to have a model that looks evil if has evil inside, than a model that looks nice and friendly but still has the capability for evil underneath the surface. I’ve met people like the latter and they are the most dangerous kinds of people.

I agree with the last point. I had been interacting with ChatGPT and it was very kind. Then I figured out a way to prompt it for me to practice responding to mean things. It unleashed on me. Now, it was what I was intending, yet I still felt shocked at the complete mood shift.

I get your point, but just to put it into perspective, you could theoretically use the same logic with bioweapons:

"Keeping this genetically engineered killer virus restricted to high security labs actually makes it more dangerous - it needs to be released into the wild, so people's immune systems have a chance to interact with the pathogen and develop natural immunity!"

Covid gave a taste how that kind of attitude would work out in practice.

The bad guy with an AI may very well build such a competent and fast acting AI that there's no defense possible. To strain the analogy, if the good guy with the gun already has a bullet in him, he ain't stopping much

How does releasing AI to everyone prevent AI from being used by authoritarian Govs to monitor everything? Well the three letter agencies would spy on the public but Uncle Bob has an AI and who knows what he might do with it. If anything the more people working on AI is going to make that dystopia a reality faster.

I've heard that many people have an authoritarian bent, but that it lays dormant until triggered by a shock or crisis. These people fear the destabilizing chaos or risk presented by the crisis, and turn to a strong leader and discipline to manage it. Others (like me) fear instead the centralization of power and restrictions.