https://en.wikipedia.org/wiki/Prisoner's_dilemma#The_iterate...
best imho, 10 min and done: https://www.youtube.com/watch?v=BOvAbjfJ0x0
longer and more over produced: https://radiolab.org/episodes/104010-one-good-deed-deserves-...
totally academic : https://onlinelibrary.wiley.com/doi/10.1111/j.1420-9101.2011...
It only takes one.
In humans v. AI, there may be only one game, if we don’t figure this out.
Search "Graham's hierarchy of disagreement" which is popular on HN. Your current criticism is at the "ad hominem" level.
It is entirely beside the point what Elon Musk's motivation is. The question is 1) if the risks referenced in the letter are real, which they certainly are. There has been informed thought on this for more than a decade. Recent LLMs have made the dangers even more clear. 2) is the letter's proposal of a 6 month pause useful. It's the most useful and plausible step I can think of. We need to take stock. It's not up to a handful of researchers to endanger the lives of 6B other people. 3) Is it actually plausible that we could get such a pause. We'll see. I doubt it but also search "death with dignity" by Yudkowsky.
Back to Musk as an "authority", the headline is about him but more than 1000 other concerned parties have signed. I will sign. Their signature vetting process is stalled because to the volume of signatures, otherwise there would be many more.
Allow me to rephrase. I am deeply concerned that there is a possibility that some of the powerful parties backing this may be trying to catch up and using an enforced or agreed to "ceasefire" as a means to catch up in capability. I also worry that some may be able to use political strong arming to accomplish this as a means of unfair competition.
> It is entirely beside the point what Elon Musk's motivation is.
Is it always beside the point what anyone's motivation is? Motivation matters.
> if the risks referenced in the letter are real, which they certainly are.
Your opinion.
> There has been informed thought on this for more than a decade. Recent LLMs have made the dangers even more clear.
> 2) is the letter's proposal of a 6 month pause useful.
More opinions.
> It's the most useful and plausible step I can think of. We need to take stock. It's not up to a handful of researchers to endanger the lives of 6B other people. 3) Is it actually plausible that we could get such a pause. We'll see. I doubt it but also search "death with dignity" by Yudkowsky.
All of this is your opinion.
> Back to Musk as an "authority", the headline is about him but more than 1000 other concerned parties have signed.
I didn't even mention Musk. I have several other names in mind. Lots of folks with AI companies (including LLMs!) that "may" be experiencing FOMO and sensing a strategy here. Maybe. Hypothetically. In a non-"ad hominem" way.
> I will sign. Their signature vetting process is stalled because to the volume of signatures, otherwise there would be many more.
People are starting counter petitions, which I'll gladly sign. The one by Suhail posted on Twitter was hilarious af.
I'll also go to whatever country doesn't "pause", because this tech is literally the most exciting development of my lifetime. And I want to spend my life doing something that matters instead of gluing distributed systems together to process financial transactions.
Also can we agree "It's not up to a handful of researchers to endanger the lives of 6B other people."?
I want to work on it too for the same reason! Can it be done safely, is worth investigating. (Yes, opinion)
Some will welcome it.
I really don't see AI research halting. Slowing a little, maybe, but I'm not sure if slowing it down a little and driving it underground will help.
So there is a benefit beyond just AGI safety.
The other one is all of us, the entire species and the future of it, gets f'cked.
Its a distraction not even worth the noise.
But, yeah, I can totally believe that our elites would prefer a solution that doesn't require a major reconfiguration of the economic system from which they benefit so much.
It's not just a potential weapon, but a potential source of untold riches and power.
That's going to be irresistible to anyone with an interest in either (ie. most politicians, governments, billionaires and corporations... and most ordinary people too, if they only had the means).
Perhaps this IS the solution to the Fermi Paradox.
LLMs are inference models.
But ok. Let's dig into this
"We can apply this concept of suspicion to the phenomenon in the field of modern artificial intelligence research. In AI research, some researchers believe that humans should pause the development of AI technology because of the potential unforeseeable risks it may bring."
An analogy would be that some of the scientists involved with the Manhattan project (see Szilárd petition) attempted to file an official protest against the use of nuclear weapons against civilian population and arguing that the weapons constructed were too dangerous to be a part of an arms race and should be placed under an international control body.
"However, another group of researchers believe that we should not stop because we cannot be certain whether those who are trying to catch up with us will use the advancement of AI technology as a hidden weapon."
Researchers are not decision makers in this or any other scenario. AI technology as a weapon? Sure but unless you are Bombs Away LeMay circa 1961 this really isn't a winnable or even useful use case. We're still in a blue sky phase. The fun thing about nuclear weapons is that if any nation feels that another is pulling too far ahead they can be used as a way to push for bi/multilateral discussions and eventually treaties limiting the production and/or deployment of weapons that risk of increasing tensions/lethality and so on . (see SALT I/II and START etc)
"This suspicion leads to a chain reaction, where people continue to worry about each other's motives and abilities, making it difficult for all parties to establish trust. Therefore, although there are good intentions, such proposals are unlikely to be implemented."
Again... read Schelling... this is just nonsense.
https://philosophybear.substack.com/p/position-statement-on-...
https://babylon5.fandom.com/wiki/Vorlon
>Distant Past
>The Vorlons were among the first races in the galaxy to attain sentience and become space-faring creatures. In the distant past, the Vorlons attempted a grand experiment into reaching a higher plane of existence where the souls existed and all life was founded. To accomplish this, they poured all their knowledge into creating a gateway to this dimension. After activating the gate, the vortex into Thirdspace was formed and the aliens within that realm came pouring out, intent on destroying all other forms of life. After much hardship, the Vorlons finally managed to push the enemy back, but one of the gateways was lost in Hyperspace where it was hidden from the Vorlons. They would encode some of their servants with an automated message to warn the younger races of the threat from the inhabitants of Thirdspace.
We can apply this concept of suspicion to the phenomenon in the field of modern artificial intelligence research. In AI research, some researchers believe that humans should pause the development of AI technology because of the potential unforeseeable risks it may bring. However, another group of researchers believe that we should not stop because we cannot be certain whether those who are trying to catch up with us will use the advancement of AI technology as a hidden weapon. This suspicion leads to a chain reaction, where people continue to worry about each other's motives and abilities, making it difficult for all parties to establish trust. Therefore, although there are good intentions, such proposals are unlikely to be implemented.