News for hackers. Stuff that matters.
Re:If AI scaling is to be shut down, let it be for a coheren... by mattmaroon on Friday March 31, 2023 @10:56AM (#35388067)
It seems as though Scott just rejects the idea of the singularity entirely. If an AI gets advanced enough to improve itself, it seem entirely reasonable that it would go from laughable to godlike in a week. I don’t know if the singularity is near, inevitable at some point, or really even possible, but it does seem like something that at least could happen. And if it occurs, it will look exactly as what he describes now. One day it’ll seem like a cool new tool that occasionally says something stupid and the next it’ll be 1,000 times smarter than us. It won’t be as we are to orangutans though, it’ll be as we are to rocks.

The six month pause though, I don’t think would be helpful. It is hubris to think we could control such an AI no matter what safeguards we try to add now. And since you couldn’t possibly police all of this activity it just seems silly to think a six month pause would do anything other than give companies that ignore it an advantage.

Re:If AI scaling is to be shut down, let it be for a coheren... by hbosch on Friday March 31, 2023 @11:03AM (#35388163)
>it seem entirely reasonable that it would go from laughable to godlike in a week.

Not if it has a power cord.

Re:If AI scaling is to be shut down, let it be for a coheren... by freejak on Friday March 31, 2023 @11:27AM (#35388551)
It just needs to convince once human to connect it to the grid which wouldn't be a difficult feat for a super-intelligent AI.
Re:If AI scaling is to be shut down, let it be for a coheren... by mattmaroon on Friday March 31, 2023 @11:39AM (#35388753)
Or just distribute itself to computers everywhere.
Re:If AI scaling is to be shut down, let it be for a coheren... by jmull on Friday March 31, 2023 @12:46PM (#35389736)
> If an AI gets advanced enough to improve itself, it seem entirely reasonable that it would go from laughable to godlike in a week.

I see this stated a lot these days but it's not true.

You're imagining a exponential phenomenon, improvement leads to greater improvement, leading to still greater improvement, etc.

However, all exponential phenomenon require the right environment to sustain and, by their nature, consume that environment. Thus, they are inherently limited in scope and duration.

The bacteria in the Petri dish grows an exponential rate... until it reaches the edge of the dish and consumes the nutrients carefully places on it. The dynamite explodes for an instant and then stops exploding once the nitroglycerin is consumed.

Also, this is an especially unconcerning scenario because (1) we haven't seen step 1 of this process yet; (2) there's no particular reason to believe the environment necessary environment to sustain the exponential growth of AI is in place (if it is, it's random chance, and therefore very likely to fizzle out almost immediately).

This is a fine sci-fi scenario, but doesn't make sense in real life.

Re:If AI scaling is to be shut down, let it be for a coheren... by bigtex88 on Friday March 31, 2023 @12:51PM (#35389807)
Sydney Bing taught itself to play chess without ever explicitly being told to learn chess. So yes, (1) is already occurring. GPT-4 displays emergent capabilities, one of which is generalized "learning".
Re:If AI scaling is to be shut down, let it be for a coheren... by jmull on Friday March 31, 2023 @01:54PM (#35390650)
There has to be a chain reaction for the proposed exponential growth.

Chatgpt3.5 would have had to be capable of creating chatgpt4, which itself is capable of creating a better chatgpt5.

So, no, (1) has not occurred yet.

We’re talking about igniting the atmosphere when no one has invented gunpowder yet.

Re:If AI scaling is to be shut down, let it be for a coheren... by mattmaroon on Friday March 31, 2023 @07:46PM (#35395323)
That’s the singularity. Nobody thinks it had already occurred. You’re basically arguing that what I said might happen won’t happen until it happens.
Re:If AI scaling is to be shut down, let it be for a coheren... by adastra22 on Friday March 31, 2023 @02:21PM (#35391130)
So why aren't we all paperclips?
Re:If AI scaling is to be shut down, let it be for a coheren... by mattmaroon on Friday March 31, 2023 @06:17PM (#35394456)
If bacteria were suddenly smarter than humans, and could instantly communicate with all the other other bacteria, plus humans one would have to assume they could start building their own petri dishes or getting us to do it for them. Especially with profit motive.

I did not claim this is a near term risk, though I’m also not sure it isn’t. But how far off is it? How can we be sure?

My real point though is that it’s either impossible or inevitable. If it can happen it will, just as the odds of global thermonuclear war are 100% over a long enough timeline.

And if it happens, this is exactly what it’ll look like. Some people will be warning about it. Some people will say it’s impossible or very far away. And then it will happen so far nobody will have had time to adjust.

Re:If AI scaling is to be shut down, let it be for a coheren... by jmull on Friday March 31, 2023 @06:54PM (#35394876)
> My real point though is that it’s either impossible or inevitable.

That’s always been true of every possibility anyone’s ever conceived. You’re just describing the general nature of reality, which is interesting, IMO, but not particularly relevant here and now.

Re:If AI scaling is to be shut down, let it be for a coheren... by mattmaroon on Friday March 31, 2023 @07:43PM (#35395297)
Lots of things are possible but not inevitable.
Re:If AI scaling is to be shut down, let it be for a coheren... by adastra22 on Friday March 31, 2023 @02:22PM (#35391146)
> it seem entirely reasonable that it would go from laughable to godlike in a week

Do you realize how ridiculous this is?