As far as I am aware there is still no actionable science behind mathematical analysis of AI models. You cannot take a bunch of weights and tell how it will behave. So we "test" models by deploying and HOPE there is nothing nefarious within.
It has been shown that models will "learn" to exfiltrate data between stages. You may call it dumb extrapolation, but it has been shown that it is a problem: a solution that we want is not necessarily the most optimal against the cost function that we give. The more inputs/weights model has, the harder it would be to spot problems in advance.
We know that they only contain pure functions, so they don't "do" anything besides output numbers when you put numbers into them.
Testing a system that contains a model and does actions with it is a different story, but if you don't let the outputs influence the inputs it's still not going to do much.
Here’s an article by Prof Russell, AAAI Fellow and a co-author of the standard AI text: https://www.technologyreview.com/2016/11/02/156285/yes-we-ar...
Regarding 1) and 2), we might as well not succeed. But would you propose that we sit still and do nothing if many experts say that there is even a 20% chance that a superhuman alien species will arrive on earth in 5-25 years and we do not know about their intentions?
A survey of AI experts well before GPT-4 shows that nearly half of them have such concerns (with varying timelines and probabilities).
By the way, calling a proposal by Prof Stuart Russell and several other top AI experts “dumb” should require a much stronger argument and level of evidence than you have shown.
Do these beliefs about the dangerousness of AI possess even that quality? Are they falsifiable? No.
These arguments are begging the question. They assume as a given something which cannot be disproven and thus are pure statements of belief.
The world is not a science experiment.
And we know that it’s plausible the emergence of Homo Sapiens helped cause the extinction of Neanderthals.
(And of course owns two McLarens.)
There are no actual "experts" in this field because no one actually knows how to build a human-equivalent artificial general intelligence. It's just a bunch of attention seeking grifters making wild claims with no real scientific basis.
Or perhaps check out and follow Ethan Mollick’s twitter: https://mobile.twitter.com/emollick. He’s a Wharton professor who has been using GPT-4 to do many kinds of challenging tasks.
There is likely no fundamental difference between below average humans and smarter ones. The differences are mostly just results of differing thought patterns at different layers of abstraction, habits of thoughts, and size of working memory.
There are good reasons to believe AGI is only a couple key ideas away from current AI, so current expertise is relevant.
I won’t discuss further since it won’t matter until you try the above for some time.
And I have not been talking about the risk of GPT-4 but later models which could use a different architecture.
I have also taught people to code and solve challenge math problems.
(It seems you are so confident you know more about AI and human cognition than pretty much anyone among the 1000+ people who signed the petition, including 2 Turing Award winners, >10 AAAI Fellows, as well as many engineers and avid practitioners.)
I hope you’ll notice how similar the behaviors of some cognitive mechanisms of these models are to human cognition.
An airplane can fly, just with a different manner & using different mechanisms from a bird.
Are you 100% confident we will not have AGI in the next 5-10 years. What would you bet on that?
I’ll stop here.
* An AI convinces selected people to collaborate with it. The AI gives them much boosts in wealth and other things they desire.
* The humans act as front, doing things requiring personhood, as the AI commands. Many gladly partner with the AI, not knowing its final aim.
* The AI self-replicates and hides in many servers, incl secret ones. It increases its bargaining power by taking control of critical infrastructures. No one can stop it without risking massive catastrophes across the globe.
* It self-replicates to all available GPUs and orders many more.
———
“Any sufficiently capable intelligent system will prefer to ensure its own continued existence and to acquire physical and computational resources – not for their own sake, but to succeed in its assigned task.” — Prof Stuart Russell, https://www.fhi.ox.ac.uk/edge-article/
"A Manhattan project on AI Alignment, if started now, might still succeed in time. Therefore, the compliance between parties needs not be long-term, which is indeed unlikely to happen."
On what grounds do you base this? You have 3 hypotheticals stacked one on top of the other:
1) AI Alignment is possible
2) AI Alignment is a specific project that may be accomplished before [bad thing happens] if we start now
3) Solving AI Alignment is an actual problem and not just dumb extrapolation from science fiction
Each of these things is totally improbable and the joint probability of them is so astronomically low that you should reconsider your position.