News for hackers. Stuff that matters.
Re:Tinygrad: Hacked 4090 driver to enable P2P by the8472 on Friday April 12, 2024 @11:56AM (#40014343)
Instead of gesturing at flawed analogies, let's return to the actual issue at hand. Do you think that agents more intelligent than humans are impossible or at least extremely unlikely to come into existence in the future? Or that such super-human intelligent agents are unlikely to have goals that are dangerous to humans? Or that they would be incapable of pursuing such goals?

Also, it seems obvious that the standard of evidence that "AI could cause extinction" can't be observing an extinction level event, because at that point it would be too late. Considering that preventive measures would take time and safety margin, which level of evidence would be sufficient to motivate serious countermeasures?

Re:Tinygrad: Hacked 4090 driver to enable P2P by stale2002 on Friday April 12, 2024 @12:16PM (#40014570)
> Instead of gesturing at flawed analogies

No. Because they are the best way of making my point and they don't allow people to make things much more vague.

The AI doomers trump card argument is to set up vague premise after vague premise, and then slip in the magic somewhere in between all of these vague claims and ignore the specifics.

If there are 10 vague premises, I can't tell you whether it is vague premise number 6, or vague premise number 7 where the magic gets slipped in, because things are so vague that it could be either! But I do know that all the vague premises add up to an extraordinary claim that requires extraordinary evidence.

So for every single argument that you are making, my thought process is going to be as follows.

"Imagine someone told me that pandora's box is real. From Greek mythology. What evidence would I require to believe that?".

Because the claims of diamond nano bot robots and mind controls humans are on the level of extraordinary cliams that are comparable to greek box mythology, even though, yes boxes in real life exist.

Phasing it in such a specific way makes my counter arguments obvious. It doesn't let you say "Well AI technology already exists. Therefore believe me", just like how someone claiming that boxes exist doesn't proves that pandora's box is serious threat.

And it make clear the central point of disagreement. The central point of disagreement, is that jumping from where we are now, to "end of the world" is an extraordinary claim that requires extraordinary evidence.

It is the gap that is large. And just pointing to existing harm with boxes or ai doesn't jump that gap.

> Also, it seems obvious that the standard of evidence that "AI could cause extinction" can't be observing an extinction level event,

Edit I have to bring this one up specifically because it is another extremely bad argument.

We don't have to observe extinction level events. There are 100 steps, way way way before that is even a possibility, that we would observe along the way.

To bring up another example, if there is an observed doomsday device at the end of the universe , there many many years of incremental steps of space travel that we will observe in real life, before we have to worry about a terrorist today building their own universe traveling space ship to fly to that doomsday device.

Same applies to AI. We would observe lots of existing capabilities before we invent world ending magic.

> which level of evidence would be sufficient to motivate serious countermeasures?

Existing real life harm, measurable in the real world, that would justify regulations for that existing harm, regardless of the world ending magic.

We would regulate AI, due to regular normal people problems, long before anyone has to bring up the magic.