News for hackers. Stuff that matters.
Re:Tinygrad: Hacked 4090 driver to enable P2P by dvdkon on Friday April 12, 2024 @09:24AM (#40012478)
I consider the burden of proof to fall on those proclaiming AGI to be an existential threat, and so far I have not seen any convincing arguments. Maybe at some point in the future we will have many anthropomorphic robots and an AGI could hack them all and orchestrate a robot uprising, but at that point the robots would be the actual problem. Similarly, if an AGI could blow up nuclear power plants, so could well-funded human attackers; we need to secure the plants, not the AGI.
Re:Tinygrad: Hacked 4090 driver to enable P2P by cjbprime on Friday April 12, 2024 @10:21AM (#40013236)
It doesn't sound like you gave serious thought to the arguments. The AGI doesn't need to hack robots. It has superhuman persuasion, by definition; it can "hack" (enough of) the humans to achieve its goals.
Re:Tinygrad: Hacked 4090 driver to enable P2P by stale2002 on Friday April 12, 2024 @11:20AM (#40013923)
AI mind control abilities are also on the level of an extraordinary claim, that requires extraordinary evidence.

It's on the level of "we better regulate wooden sticks so Voldemort doesn't use the imperious curse on us!".

That's how I treat such claims. I treat them the same as someone literally talking about magic from Harry potter.

There isn't nothing that would make me believe that. But it requires actual evidence and not thought experiments.

Re:Tinygrad: Hacked 4090 driver to enable P2P by the8472 on Friday April 12, 2024 @11:33AM (#40014077)
Voldemort is fictional and so are bumbling wizard apprentices. Toy-level, not-yet-harmful AIs on the other hand are real. And so are efforts to make them more powerful. So the proposition that more powerful AIs will exist in the future is far more likely than an evil super wizard coming into existence.

And I don't think literal 5-word-magic-incantation mind control is essential for an AI to be dangerous. More subtle or elaborate manipulation will be sufficient. Employees already have been duped into financial transactions by faked video calls with what they assumed to be their CEOs[0], and this didn't require superhuman general intelligence, only one single superhuman capability (realtime video manipulation).

[0] https://edition.cnn.com/2024/02/04/asia/deepfake-cfo-scam-ho...

Re:Tinygrad: Hacked 4090 driver to enable P2P by stale2002 on Friday April 12, 2024 @11:38AM (#40014136)
> Toy-level, not-yet-harmful AIs on the other hand are real.

A computer that can cause harm is much different than the absurd claims that I am disagreeing with.

The extraordinary claims that are equivalent to saying that the imperious curse exists would be the magic computers that create diamond nanobots and mind control humans.

> that more powerful AIs will exist in the future

Bad argument.

Non safe Boxes exist in real life. People are trying to make more and better boxes.

Therefore it is rational to be worried about Pandora's box being created and ending the world.

That is the equivalent argument to what you just made.

And it is absurd when talking about world ending box technology, even though Yes dangerous boxes exist, just as much as it is absurd to claim that world ending AI could exist.

Re:Tinygrad: Hacked 4090 driver to enable P2P by the8472 on Friday April 12, 2024 @11:56AM (#40014343)
Instead of gesturing at flawed analogies, let's return to the actual issue at hand. Do you think that agents more intelligent than humans are impossible or at least extremely unlikely to come into existence in the future? Or that such super-human intelligent agents are unlikely to have goals that are dangerous to humans? Or that they would be incapable of pursuing such goals?

Also, it seems obvious that the standard of evidence that "AI could cause extinction" can't be observing an extinction level event, because at that point it would be too late. Considering that preventive measures would take time and safety margin, which level of evidence would be sufficient to motivate serious countermeasures?

Re:Tinygrad: Hacked 4090 driver to enable P2P by stale2002 on Friday April 12, 2024 @12:16PM (#40014570)
> Instead of gesturing at flawed analogies

No. Because they are the best way of making my point and they don't allow people to make things much more vague.

The AI doomers trump card argument is to set up vague premise after vague premise, and then slip in the magic somewhere in between all of these vague claims and ignore the specifics.

If there are 10 vague premises, I can't tell you whether it is vague premise number 6, or vague premise number 7 where the magic gets slipped in, because things are so vague that it could be either! But I do know that all the vague premises add up to an extraordinary claim that requires extraordinary evidence.

So for every single argument that you are making, my thought process is going to be as follows.

"Imagine someone told me that pandora's box is real. From Greek mythology. What evidence would I require to believe that?".

Because the claims of diamond nano bot robots and mind controls humans are on the level of extraordinary cliams that are comparable to greek box mythology, even though, yes boxes in real life exist.

Phasing it in such a specific way makes my counter arguments obvious. It doesn't let you say "Well AI technology already exists. Therefore believe me", just like how someone claiming that boxes exist doesn't proves that pandora's box is serious threat.

And it make clear the central point of disagreement. The central point of disagreement, is that jumping from where we are now, to "end of the world" is an extraordinary claim that requires extraordinary evidence.

It is the gap that is large. And just pointing to existing harm with boxes or ai doesn't jump that gap.

> Also, it seems obvious that the standard of evidence that "AI could cause extinction" can't be observing an extinction level event,

Edit I have to bring this one up specifically because it is another extremely bad argument.

We don't have to observe extinction level events. There are 100 steps, way way way before that is even a possibility, that we would observe along the way.

To bring up another example, if there is an observed doomsday device at the end of the universe , there many many years of incremental steps of space travel that we will observe in real life, before we have to worry about a terrorist today building their own universe traveling space ship to fly to that doomsday device.

Same applies to AI. We would observe lots of existing capabilities before we invent world ending magic.

> which level of evidence would be sufficient to motivate serious countermeasures?

Existing real life harm, measurable in the real world, that would justify regulations for that existing harm, regardless of the world ending magic.

We would regulate AI, due to regular normal people problems, long before anyone has to bring up the magic.

Re:Tinygrad: Hacked 4090 driver to enable P2P by cjbprime on Friday April 12, 2024 @11:58AM (#40014373)
What do you think mind control is? Think President Trump but without the self-defeating flaws, with an ability to stick to plans, and most importantly the ability to pay personal attention to each follower to further increase the level of trust and commitment. Not Harry Potter.

People will do what the AI says because it is able to create personal trust relationships with them and they want to help it. (They may not even realize that they are helping an AI rather than a human who cares about them.)

The normal ways that trust is created, not magical ones.

Re:Tinygrad: Hacked 4090 driver to enable P2P by stale2002 on Friday April 12, 2024 @02:46PM (#40016318)
> What do you think mind control is?

The magic technology that is equivalent to the imperious curse from Harry Potter.

> The normal ways that trust is created, not magical ones.

Buildings as a technology are normal. They are constantly getting taller and we have better technology to make them taller.

But, even though buildings are a normal technology, I am not going to worry about buildings getting so tall soon that they hit the sun.

This is the same exact mistake that every single AI doomers makes. What they do is they take something normal, and then they infinitely extrapolate it out to an absurd degree, without admitting that this is an extraordinary claim that requires extraordinary evidence.

The central point of disagreement, that always gets glossed over, is that you can't make a vague claim about how AI is good at stuff, and then do your gigantic leap from here to over there which is "the world ends".

Yes that is the same as comparing these worries to those who worry about buildings hitting the sun or the imperious curse.

Re:Tinygrad: Hacked 4090 driver to enable P2P by FeepingCreature on Friday April 12, 2024 @05:39PM (#40017970)
Less than a month ago: https://arxiv.org/abs/2403.14380 "We found that participants who debated GPT-4 with access to their personal information had 81.7% (p < 0.01; N=820 unique participants) higher odds of increased agreement with their opponents compared to participants who debated humans."

And it's only gonna get better.

Re:Tinygrad: Hacked 4090 driver to enable P2P by stale2002 on Friday April 12, 2024 @09:18PM (#40019497)
Yes, and I am sure that when people do a google search for "Good arguments in favor of X", that they are also sometimes convinced to be more in favor of X.

Perhaps they would be even more convinced by the google search than if a person argued with them about it.

That is still much different from "The AI mind controls people, hacks the nukes, and ends the world".

Its that second part that is the the fantasy land situation that requires extraordinary evidence.

But, this is how conversations about doomsday AI always go. People say "Well isn't AI kinda good at this extremely vague thing Y, sometimes? Imagine if AI was infinitely good at Y! That means that by extrapolation, the world ends!".

And that covers basically every single AI doom argument that anyone ever makes.

Re:Tinygrad: Hacked 4090 driver to enable P2P by FeepingCreature on Saturday April 13, 2024 @08:25AM (#40022698)
If the only evidence for AI doom you will accept is actual AI doom, you are asking for evidence that by definition will be too late.

"Show me the AI mindcontrolling people!" AI mindcontrolling people is what we're trying to avoid seeing.

The trick is, in the world in which AI doom is in the future, what would you expect to see now that is different from the world in which AI doom is not in the future?

Re:Tinygrad: Hacked 4090 driver to enable P2P by stale2002 on Saturday April 13, 2024 @12:31PM (#40024201)
> If the only evidence for AI doom you will accept is actual AI doom

No actually. This is another mistake that the AI doomers make. They pretend like a demand for evidence means that the world has to end first.

Instead, what would be perfectly good evidence, would be evidence of significant incremental harm that requires regulation on its own, independent of any doom argument.

In between "the world literally ends by magic diamond nanobots and mind controlling AI" and "where we are today" would be many many many situations of incrementally escalating and measurable harm that we would see in real life, decades before the world ending magic happens.

We can just treat this like any other technology, and regulate it when it causes real world harm. Because before the world ends by magic, there would be significant real world harm that is similar to any other problem in the world that we handle perfectly well.

Its funny because you committing the exact mistake that I was criticizing in my original post, where you did the absolutely massive jump and hand waved it away.

> what would you expect to see now that is different from the world in which AI doom is not in the future?

What I would expect is for the people who claim to care about AI doom to actually be trying to measure real world harm.

Ironically, I think the people who are coming up with increasingly thin excuses as for why they don't have to find evidence are increasing the likelyhood of such AI doom much more than anyone else because they are abandoning the most effective method of actually convincing the world of the real world damage that AI could cause.

Re:Tinygrad: Hacked 4090 driver to enable P2P by FeepingCreature on Saturday April 13, 2024 @01:16PM (#40024504)
Well, at least if you see escalating measurable harm you'll come around, I'm happy about that. You won't necessarily get the escalating harm even if AI doom is real though, so you should try to discover if it is real even in worlds where hard takeoff is a thing.

> What I would expect is for the people who claim to care about AI doom to actually be trying to measure real world harm.

Why bother? If escalating harm is a thing, everyone will notice. We don't need to bolster that, because ordinary society has it handled.

Re:Tinygrad: Hacked 4090 driver to enable P2P by stale2002 on Saturday April 13, 2024 @05:01PM (#40026087)
> You won't necessarily get the escalating harm even if AI doom is real though

Yes we would. Unless you are one of those people who think that the magic doom nanobots are going to be invented overnight.

My comparisions to someone who is worried about literal magic, from harry potter, is apt.

But at that point, if you are worried about magic showing up instantly, then your position is basically not falsifiable. You can always retreat to some untestable, unfalsifiable magic.

Like there is actually nothing I could say, no evidence I could show to ever convince someone out of that position.

On the other hand, my position is actually fasifiable. There is absolutely all sorts of non world ending evidence that could convince me to think that AI is dangerous.

But nobody on the doomer side seems to care about any of that. Instead they invent positions that seem almost tailor made to avoid being falsifiable or disprovable so that they can continue to believe them despite any evidence to the contrary.

As in, if I were to purposeful invent an idea or philosophy that is impossible to be disproved or convinced out of the "I can't show you evidence because the world will end" position is what I would invent.

> you'll come around,

Do you admit that you won't though? Do you admit that no matter what evidence is shown to you, that you can just retreat and say that the magic could happen at any time?

Or even if this isn't you literally, that someone in your position could dismiss all counter evidence, no matter what, and nobody could convince someone out of that with evidence?

I am not sure how someone could ever possibly engage with you seriously on any of this, if that is your position.

Re:Tinygrad: Hacked 4090 driver to enable P2P by the8472 on Saturday April 13, 2024 @07:04PM (#40027020)
> Like there is actually nothing I could say, no evidence I could show to ever convince someone out of that position.

There is, it is just very hard to obtain. Various formal proofs would do. On upper bounds. On controllability. On scalability of safety techniques.

The manhattan project scientists did check whether they'd ignite the atmosphere before detonating their first prototype. Yes, that was much simpler task. But there's no rule in nature that says proving a system to be safe must be as easy as creating the system. Especially when the concern is that the system adaptive and adversarial.

Recursive self-improvement is a positive feedback loop, like nuclear chain reactions, like virus replication. So if we have an AI that can program then we better make sure that it either cannot sustain such a positive feedback loop or that it remains controllable beyond criticality. Given the complexity of the task it appears unlikely that a simple ten-page paper proving this will show up on arxiv. But if one did that'd be great.

>> You won't necessarily get the escalating harm even if AI doom is real though

> Yes we would.

So what does guarantee a visible catastrophe that won't be attributed to human operators using a non-agentic AI incorrectly? We keep scaling and the systems will be treated as assistants/optimizers and it's always the operators fault. Until we roughly reach human-level on some relevant metrics. And at that point there's a very narrow complexity range from idiot to genius (human brains don't vary by orders of magnitude!). So as far as hardware goes this could be a very narrow range and we could shoot straight from "non-agentic sub-human AI" to "agentic superintelligence" in short timescales once the hardware has that latent capacity. And up until that point it will always have been a human error, lax corporate policies, insufficient filtering of the training set or whatever.

And it's not that it must happen this way. Just that there doesn't seem anything ruling it and similar pathways out.

Re:Tinygrad: Hacked 4090 driver to enable P2P by CamperBob2 on Friday April 12, 2024 @11:14AM (#40013875)
Then it's just a matter of evolution in action.

And while it doesn't take a God to start evolution, it would take a God to stop it.

Re:Tinygrad: Hacked 4090 driver to enable P2P by hollerith on Friday April 12, 2024 @11:16AM (#40013893)
You might be OK with suddenly dying along with all your friends and family, but I am not even if it is "evolution in action".
Re:Tinygrad: Hacked 4090 driver to enable P2P by CamperBob2 on Friday April 12, 2024 @11:22AM (#40013961)
Historically governments haven't needed computers or AI to do that. They've always managed just fine.

Punched cards helped, though, I guess...

Re:Tinygrad: Hacked 4090 driver to enable P2P by FeepingCreature on Friday April 12, 2024 @05:41PM (#40017985)
gestures at the human population graph wordlessly
Re:Tinygrad: Hacked 4090 driver to enable P2P by CamperBob2 on Saturday April 13, 2024 @12:02PM (#40024010)
Agent Smith smiles mirthlessly
Re:Tinygrad: Hacked 4090 driver to enable P2P by the8472 on Friday April 12, 2024 @09:37AM (#40012693)
You say you have not seen any arguments that convince you. Is that just not having seen many arguments or having seen a lot of arguments where each chain contained some fatal flaw? Or something else?