If you think these systems are going to be no more capable than a magpie, then I think you're making a very big mistake, the sort of mistake that in our lifetime could plausible kill millions of people.

ChatGPT can already write code. A magpie cannot do that.

That's one of those capabilities that seems super scary if you truly believe that writing code is one of the most important things a human can do. Computers have, of course, been writing computer programs for a long time. Next thing you know, they'll be beating us at chess.

I think you're confusing importance with power.

Can it execute code?

It can submit the code that it's written for execution if you tell it that it can, by utilizing specific markers in the output that get processed. There already are frameworks around this that make it possible to e.g. call an arbitrary Python function as part of answering the question.

That’s an easy prediction to make; at worst you’re cautious.

And it’s a powerful tool. Even staplers have rules around their use: no stapling people, no hitting people with a stapler, don’t use a staple to pick a lock, etc.

But nobody blames the stapler, is my point.

With the advancements of AI in the past year alone it seems silly to think that, within a lifetime, AI doesn’t have the ability to manifest society collapsing contagion. AI is certainly going to be granted more network access than it currently has, and the feedback loop between AI, people, and the network is going to increase exponentially.

Reduced to the sum of its parts, the internet is less than a magpie, yet viruses and contagion of many forms exist in it, or are spread though it. ChatGPT 2.0 greatly amplifies the effects of those contagion, regardless of our notions of what intelligence or agency actually is.

Innovation doesn’t follow any path; discovery is messy. No matter how much we advance towards smaller chips, we are never going to get to 0nm, for example.

There are limits, but even if there weren’t, we’re no closer to AGI today then we were a year ago. It’s just a different thing entirely.

LLMs are cool! They’re exciting! There should be rules around their responsible operation! But they’re not going to kill us all, or invade, or operate in any meaningful way outside of our control. Someone will always be responsible for them.

> It's pants-on-head silly to think "ChatGPT 2.0" is anything other than, at best, a magpie. If you put the nuclear codes under a shiny object, or arranged it such that saying a random basic word would trigger a launch, then yeah a magpie could fire off nukes.

> But why the hell would you do that?!?!

Exactly. Except that they are doing it exactly, and we are the ones asking 'why the hell would you do that?!'.

A chatGPT 2.0 knows everything, really. If you command it (or it reaches the conclusion itself for some unrelated commands), it can hack nuclear arsenals and nuke everything, or worse.

> A chatGPT 2.0 knows everything, really. If you command it (or it reaches the conclusion itself for some unrelated commands), it can hack nuclear arsenals and nuke everything, or worse.

Pretty much everything here is incorrect. An LLM is not omniscient. An LLM does not think or reason. An LLM does not reach conclusions.

There is no "AI" here in the sense you're saying.