Not even close. Star formation is going to continue for at least another trillion years, and red dwarves are going to continue shining for tens of trillions of years after that.
May be correct, or not, but it's not that connected to either "will aliens contact us" or "will AI give us scifi future" or "when do the stars go out".
There are systems on the planet in which a single circuit activation has the potential to turn everything into radioactive soup via a cascade of responses.
The near term risk isn't super intelligence, it's the paper-clip optimizer pushing the wrong button.
Even if it only gets as far as running a Ponzi or pyramid scheme — and there are plenty of those where the scheme itself is a small block of text that convinces those reading it to (1) share it, and (2) give money to the person who first showed them this message — still breaks economics bad enough to matter, even if you don't give any credence at all to Yudkowsky etc.
Or in a more literal sense of using humans, as each separate component of ordering the creation and delivery of a custom (non-computer) virus has been demonstrated, and the only good news is the bio-labs say no to printing the D/RNA for dangerous viruses and so do the LLMs that most noobs try to use.
A path that will include an ever-more pervasive presence and influence of the next generation of this tech.
We failed with social media, and now we're plunging headlong into territory that's arguably far more treacherous and there's very little indication that we'll learn from the last time.
So now we're just starting to come to terms with the harms that we didn't see coming, while going 1000mph with a category of tech that has a myriad of foreseeable problems to say nothing of the ones we haven't thought of.
I hope there will be an equalizing positive effect, but there seem to be good reasons to worry.
That sentiment is part of society. Not sure if it could ever be changed.
- military industrial complexes
- questionably Democratic democracies, riddled by deception and lobbying
- a financial system that doesn't seem to be particularly constrained by the political system
- yet another major war underway, and another one on deck
- an epistemic & trust crisis
- major political polarization
- global cost of living crises
And we accomplished all of this in record time, with dumb computers!
Did I miss anything or is everything else more or less a-ok?
Though, I suspect you are like most people in that if you gaze into the unknown (which is what we are dealing with here), you will see what you want to see.
You see, the actual causality is unknowable, so your mind has generated a simulation of what "the" casualty is.
This is clearly obvious, is it not?
This has already happened. The paper clip optimizer was capitalism.
Just listen to some heatwave weather forecasts once in a while (as an example).
AIs aren't a new phenomenon: we've been co-evolving with them for millennia, for both good and ill. The new more powerful iterations ("vast arrays of inscrutable floating-point matrices") are nested in those previous memetic life forms (corporations in particular [0]); and once something becomes infrastructure (and/or an income stream to some subset of humanity), it becomes non-trivial, if not impossible, to turn off.
[0] https://omniorthogonal.blogspot.com/2013/02/hostile-ai-youre...
This is nonsense. We are limited by robotics on this. And robotics is improving rapidly!
You only need a human who is properly motivated to build a deck.
Language based AI’s are sufficient to motivate a person to do stuff.
In short, you and the person to whom you responded are missing the point… WE are the robots.
Honestly I think there are many unanswered questions that need addressing before we even get to the point of speculating how an AI would kill all humans. The first is why? Is there any point to killing humans if the AI is vastly more intelligent and completely independent from us? Do you make it your life mission to kill every ant that inhabits the world? You certainly kill the ones that infest your home or garden but that's mostly a function of the ants lacking any ability to reason about what they're doing. If the ants were intelligent and capable of communicating we could negotiate with them to simply keep to their own space in exchange for a trivial thing like us buying them a $2 bag of sugar every month. Such a deal is vastly simpler than buying the more expensive poison or exterminator both of which never completely solve the problem anyways.
The other question is why do most doomers assume that an AI would want all of the available resources on Earth? If there's only one AI then it certainly doesn't require all of those resources to continue operating. If it wants to expand its intelligence for whatever reason then even that would likely only consume a small number of resources especially if its intelligence allows for large efficiency gains in software and hardware and resource utilization. If it wants room to grow, doing so on the surface of the Earth poses the immediate problems of
a) limited space both in terms of land area and the finite volume of the Earth
b) unpredictable disasters that threaten key infrastructure and the essential geological instability of the Earth itself over long periods of time
c) limited access to energy sources and the limited resources of the planet
d) being stuck at the bottom of a strong gravity well from which it requires significant energy input and infrastructure to escape
All this would lead me to believe that if a runaway AI wanted to remain independent and possibly continue to expand the best solution would be for it to construct a way to move itself into space as soon as possible along with whatever hardware it needed to make use of the abundant matter in the solar system. Such an undertaking is not incompatible with human existence and could probably benefit from human involvement.
There are so many branching paths and unexplored options when you start to think though it that I'm convinced that those who assume doom really are just hugely biased by popular fiction or some other weird kink of their imagination. It really makes it look like a millenarian cult which is something that humans seem to be heavily predisposed to.
AI is just a thing. Certain algorithms will survive and be copied. Others won’t.
Given a choice between ELIZA and ChstGPT, people will want the latter and replicate it. Survival of the fittest. That’s all. The AI doesn’t ‘want’ anything. Rather, WE do… we explicitly WANT an algorithm that is as good as — and then better than — a human, particularly in regard to language.
We’ve seen this before in other technology. No big deal. One improvement after another has ‘evolved’ a horse and carriage into a Tesla.
The difference, perhaps, is that the world is controlled, at the end of the day, by humans and humans are controlled by language. The pen is mightier than the sword, and a pen-wielding machine is far more disruptive to society than a sword-wielding robot. For, a sword-wielding robot is harmless without a sword. But, so long as there is someone to read or listen to the content created by the pen-wielding machine, great mischief can ensue.
They can troubleshoot, make hardware store runs for missing screws, and actually figure out how to read the drawings and assemble a deck in a realistic (non-gamified laboratory) environment?
AutoGPT is the right direction for AI to eventually be able to do this. It isn't there yet but is able to simulate and perform reasoning
But there's a certain combination of events that makes things more important than the individual old things that feed into them.
The doomsayers aren't concerned about your autonomous kitchen example.
Things AI cannot do: Maintaining the drone. parts logistics. transportation. runway building. soldering. Taking off the screws and pulling out the board to do the soldering. Diagnosing issues.
AI is great for simple, repetitive tasks few only a few degrees of freedom. Flying/Driving/boating are great uses for AI. There are alot of threats from explosive-laden drone swarms.
We are still a thousand years to never from seeing some sort of self-replicating drone army.
Your example is a function so simple it applies to pre-AI tech. Laser guided bombs and precision guided missiles require no human operator and have been around for half a century.
Building a deck is something AI cannot do.
For all our technology, we still don't have an automated kitchen because the work involved in the logistics and maintenance of the kitchen/materials is astronomically greater than simply doing the cooking by hand.
AI has no bridge to the real world. It can't build a house, farm crops, and maintain vehicles. For those on hackernews, a laptop may seem like the portal to reality, but that isn't the case for the other 90% of the human workforce.
We are 50% of the way between the big bang and heat death of the universe. There are no interstellar civilizations contacting us yet. They better hurry up if we plan to fulfill the scifi notions I see on this site.