I don’t care to engage in these nitpicky hypotheticals. We all know what I’m saying.
If the scientists building these systems aren’t credible and the commentators not building these systems aren’t credible, then who is?
If the answer is “no one,” okay, cool, that inscrutability is another reason for, not against, caution.
See grey goo: https://en.wikipedia.org/wiki/Gray_goo
How about nukes and atmospheric fire: https://www.insidescience.org/manhattan-project-legacy/atmos...
> scientists ... credible
The science ends at the math. Go play with an LLM at home. Run it on your GPU, see what it does, what its limits are. Realize that it's a system that never "grows", that lacking noise it becomes deterministic. Once you look behind the curtain and find out what's there, a lot of the hype to call it AI looks dumb.
Yes, LLMs currently don't learn. This is also true for humans who cannot form new memories. A human who cannot learn anything new besides the contents of their short-term memory is at a huge disadvantage, but (hopefully) no less capable if given access to a scratchpad and more thinking time to compensate.
As for the part that they are deterministic lacking noise... Why would you not provide it with noise? Generating randomness is something that modern machines are quite good at, so there isn't much of a reason not to inject it at inference time.
I mean we are talking about future existential threats to mankind and the best response would be 'trust me bro'?
Sorry but trust in such matters is not something given freely nor easily. People are generally pretty conservative when their existence may be in stake. Not so much those whose income is tightly coupled with pushing things fast and basically ignoring those concerns. This is where SV's 'push fast and break things' hits a titanium wall of general mankind.
Or maybe folks can be bribed / attention diverted with endless stream of photorealistic cat videos, what do I know.
I have no clue whether LLMs will get us there (neither do you) and I have no clue whether we’re 6 months away or 600 years away (neither do you).
What LLMs have revealed is that very reasonable checkpoints will be blown through for commercial purposes. We are reliably producing unexpected emergent capabilities with each new scale up. We are reliably failing to get these systems to behave as their creators intend. We are continually deploying these systems to the public anyway, and we are continually connecting them to other, increasingly vital, systems.
If your argument is “we’ll never reach a superhuman or self-improving AI,” I’m open to hearing that argument. Tell me how you know. If nothing precludes us from building it, then I know that we will eventually build it due to the commercial incentives.
If your argument is “we’ll get there, but don’t worry because we’ll stop minutes before deployment and figure out all the alignment/emergent behaviors/social distribution/airgapping problems before pressing ‘deploy’,” then I ask what is your evidence? What capability have we shown to make such a decision at such a moment?
Congratulations, you're performing one of the major functions of intellect that an LLM can't. Learning. You aren't a deterministic system. LLM's aren't a model that is suddenly going to change all the past failures of "make it bigger and AGI will emerge from it"... At its core what an LLM is produces something interesting, but it's NOT a path to intellect... It's more or less turning language into a mathematical model. Are we so foolish, so hubris driven that something bigger will emerge from the digitized version of our language, and our vision alone, we made god in our image and how did THAT work out?
And if a super intelligent AI emerges tomorrow so what? Is it going to be able to manipulate us all based on a deep understand of people, that it gained from the piles of bad psych research? Is it going to melt down reactors and launch nukes, cause we live in a brittle world and that is suicide for your AGI (and improbable to impossible). Is it going to kill us with paperclips... cause we're so stupid we keep feeding it metal till we drown ourselves.
If AGI shows up tomorrow how many decades to a tipping point where it could "take over" where it could "cause harm"? Walk me through that process where the world is automated enough that paperclips are what kills us. That AGI is gonna be stuck in a cradle, a data center for a LONG time. It's gonna need to be fed power, hardware, water (for cooling). A can of gas and a bag of rags will stop, A few people with shovels or one person with a backhoe cuts it off from the world.
It takes 50 to 100 years for that AGI to get to a place where it does not need us at all. I would be more worried about a doomer mudering it in its cradle than it going off the rails in that time frame.
What then?
It exists in a physical (if distributed) location and requires inputs and outputs in electric cables and fibre cables.
Destroy the brain.
The most practiced researchers of alchemy were convinced that they could turn lead into gold. By itself, this argument is unconvincing. When it comes to the potential for fabulous wealth and/or unimaginable power, incentives distort and people are inclined to abandon their scruples.