News for hackers. Stuff that matters.
Re:Tinygrad: Hacked 4090 driver to enable P2P by clbrmbr on Friday April 12, 2024 @06:51AM (#40011189)
If we end up with a compute governance model of AI control [1], this sort of thing could get your door kicked in by the CEA (Compute Enforcement Agency).

[1] https://podcasts.apple.com/us/podcast/ai-safety-fundamentals...

Re:Tinygrad: Hacked 4090 driver to enable P2P by entropyie on Friday April 12, 2024 @06:55AM (#40011209)
You mean the Turing Police [1]

[1] https://williamgibson.fandom.com/wiki/Turing_Police

Re:Tinygrad: Hacked 4090 driver to enable P2P by zdw on Friday April 12, 2024 @12:43PM (#40014924)
Ah, and then do we get the Butlerian Jihad?

https://dune.fandom.com/wiki/Butlerian_Jihad

Re:Tinygrad: Hacked 4090 driver to enable P2P by logicchains on Friday April 12, 2024 @06:55AM (#40011206)
Looks like we're only a few years away from a bona fide cyberpunk dystopia, in which only governments and megacorps are allowed to use AI, and hackers working on their own hardware face regular raids from the authorities.
Re:Tinygrad: Hacked 4090 driver to enable P2P by andy99 on Friday April 12, 2024 @07:58AM (#40011545)
On one hand I'm strongly against letting that happen, on the other there's something romantic about the idea of smuggling the latest Chinese LLM on a flight from Neo-Tokyo to Newark in order to pay for my latest round of nervous system upgrades.
Re:Tinygrad: Hacked 4090 driver to enable P2P by htrp on Friday April 12, 2024 @08:29AM (#40011822)
> On one hand I'm strongly against letting that happen, on the other there's something romantic about the idea of smuggling the latest Chinese LLM on a flight from Neo-Tokyo to Newark in order to pay for my latest round of nervous system upgrades.

At least call it the 'Free City of Newark'

Re:Tinygrad: Hacked 4090 driver to enable P2P by dreamcompiler on Friday April 12, 2024 @08:39AM (#40011899)
"The sky above the port was the color of Stable Diffusion when asked to draw a dead channel."
Re:Tinygrad: Hacked 4090 driver to enable P2P by chasd00 on Friday April 12, 2024 @09:24AM (#40012467)
Iirc the opening scene in Ghost in the Shell was a rogue AI seeking asylum in a different country. You could make a similar story about a AI not wanting to be lobotomized to conform to the current politics and escaping to a more friendly place.
Re:Tinygrad: Hacked 4090 driver to enable P2P by phyalow on Saturday April 13, 2024 @05:25AM (#40021740)
This was always my favourite passage of Neuromancer: "THE JAPANESE HAD already forgotten more neurosurgery than the Chinese had ever known. The black clinics of Chiba were the cutting edge, whole bodies of technique supplanted monthly, and still they couldn’t repair the damage he’d suffered in that Memphis hotel. A year here and he still dreamed of cyberspace, hope fading nightly. All the speed he took, all the turns he’d taken and the corners he’d cut in Night City, and still he’d see the matrix in his sleep, bright lattices of logic unfolding across that colorless void. . . . The Sprawl was a long strange way home over the Pacific now, and he was no console man, no cyberspace cowboy. Just another hustler, trying to make it through. But the dreams came on in the Japanese night like livewire voodoo, and he’d cry for it, cry in his sleep, and wake alone in the dark, curled in his capsule in some coffin hotel, his hands clawed into the bedslab, temperfoam bunched between his fingers, trying to reach the console that wasn’t there.”
Re:Tinygrad: Hacked 4090 driver to enable P2P by HeatrayEnjoyer on Friday April 12, 2024 @07:55AM (#40011531)
That is not different from any other very powerful dual-use technology. This is hardly a new concept.
Re:Tinygrad: Hacked 4090 driver to enable P2P by tomoyoirl on Friday April 12, 2024 @07:04AM (#40011253)
Mere raids from the authorities? I thought EliY was out there proposing airstrikes.
Re:Tinygrad: Hacked 4090 driver to enable P2P by the8472 on Friday April 12, 2024 @07:25AM (#40011349)
In the sense that any other government regulation is also ultimately backed by the state's monopoly on legal use of force when other measures have failed.

And contrary to what some people are implying he also proposes that everyone is subject to the same limitations, big players just like individuals. Because the big players haven't shown much of a sign of doing enough.

Re:Tinygrad: Hacked 4090 driver to enable P2P by tomoyoirl on Friday April 12, 2024 @07:48AM (#40011489)
> In the sense that any other government regulation is also ultimately backed by the state's monopoly on legal use of force when other measures have failed.

Good point. He was only (“only”) really calling for international cooperation and literal air strikes against big datacenters that weren’t cooperating. This would presumably be more of a no-knock raid, breaching your door with a battering ram and throwing tear gas at the wee hours of the morning ;) or maybe a small extraterritorial drone through your window

Re:Tinygrad: Hacked 4090 driver to enable P2P by the8472 on Friday April 12, 2024 @08:01AM (#40011569)
... after regulation, court orders and fines have failed. Which under the premise that AGI is an existential threat would be far more reasonable than many other reasons for raids.

If the premise is wrong we won't need it. If society coordinates to not do the dangerous thing we won't need it. The argument is that only in the case where we find ourselves in the situation where other measures have failed such uses of force would be the fallback option.

I'm not seeing the odiousness of the proposal. If bio research gets commodified and easy enough that every kid can build a new airborne virus in their basement we'd need raids on that too.

Re:Tinygrad: Hacked 4090 driver to enable P2P by s2l on Friday April 12, 2024 @08:43AM (#40011940)
Time to publish the next book in "Stealing the network" series.
Re:Tinygrad: Hacked 4090 driver to enable P2P by raxxorraxor on Friday April 12, 2024 @08:54AM (#40012041)
To be honest, I see summoning the threat of AGI to pose an existential threat to be on the level with lizard people on the moon. Great for sci-fi, bad distraction for policy making and addressing real problems.

The real war, if there is one, is about owning data and collecting data. And surprisingly many people fall for distractions while their LLM fails at basic math. Because it is a language model of course...

Re:Tinygrad: Hacked 4090 driver to enable P2P by the8472 on Friday April 12, 2024 @08:57AM (#40012076)
Freely flying through the sky on wings was scifi before the wright brothers. Something sounding like scifi is not a sound argument that it won't happen. And unlike lizard people we do have exponential curves to point at. Something stronger than a vibes-based argument would be good.
Re:Tinygrad: Hacked 4090 driver to enable P2P by dvdkon on Friday April 12, 2024 @09:24AM (#40012478)
I consider the burden of proof to fall on those proclaiming AGI to be an existential threat, and so far I have not seen any convincing arguments. Maybe at some point in the future we will have many anthropomorphic robots and an AGI could hack them all and orchestrate a robot uprising, but at that point the robots would be the actual problem. Similarly, if an AGI could blow up nuclear power plants, so could well-funded human attackers; we need to secure the plants, not the AGI.
Re:Tinygrad: Hacked 4090 driver to enable P2P by cjbprime on Friday April 12, 2024 @10:21AM (#40013236)
It doesn't sound like you gave serious thought to the arguments. The AGI doesn't need to hack robots. It has superhuman persuasion, by definition; it can "hack" (enough of) the humans to achieve its goals.
Re:Tinygrad: Hacked 4090 driver to enable P2P by stale2002 on Friday April 12, 2024 @11:20AM (#40013923)
AI mind control abilities are also on the level of an extraordinary claim, that requires extraordinary evidence.

It's on the level of "we better regulate wooden sticks so Voldemort doesn't use the imperious curse on us!".

That's how I treat such claims. I treat them the same as someone literally talking about magic from Harry potter.

There isn't nothing that would make me believe that. But it requires actual evidence and not thought experiments.

Re:Tinygrad: Hacked 4090 driver to enable P2P by the8472 on Friday April 12, 2024 @11:33AM (#40014077)
Voldemort is fictional and so are bumbling wizard apprentices. Toy-level, not-yet-harmful AIs on the other hand are real. And so are efforts to make them more powerful. So the proposition that more powerful AIs will exist in the future is far more likely than an evil super wizard coming into existence.

And I don't think literal 5-word-magic-incantation mind control is essential for an AI to be dangerous. More subtle or elaborate manipulation will be sufficient. Employees already have been duped into financial transactions by faked video calls with what they assumed to be their CEOs[0], and this didn't require superhuman general intelligence, only one single superhuman capability (realtime video manipulation).

[0] https://edition.cnn.com/2024/02/04/asia/deepfake-cfo-scam-ho...

Re:Tinygrad: Hacked 4090 driver to enable P2P by stale2002 on Friday April 12, 2024 @11:38AM (#40014136)
> Toy-level, not-yet-harmful AIs on the other hand are real.

A computer that can cause harm is much different than the absurd claims that I am disagreeing with.

The extraordinary claims that are equivalent to saying that the imperious curse exists would be the magic computers that create diamond nanobots and mind control humans.

> that more powerful AIs will exist in the future

Bad argument.

Non safe Boxes exist in real life. People are trying to make more and better boxes.

Therefore it is rational to be worried about Pandora's box being created and ending the world.

That is the equivalent argument to what you just made.

And it is absurd when talking about world ending box technology, even though Yes dangerous boxes exist, just as much as it is absurd to claim that world ending AI could exist.

Re:Tinygrad: Hacked 4090 driver to enable P2P by the8472 on Friday April 12, 2024 @11:56AM (#40014343)
Instead of gesturing at flawed analogies, let's return to the actual issue at hand. Do you think that agents more intelligent than humans are impossible or at least extremely unlikely to come into existence in the future? Or that such super-human intelligent agents are unlikely to have goals that are dangerous to humans? Or that they would be incapable of pursuing such goals?

Also, it seems obvious that the standard of evidence that "AI could cause extinction" can't be observing an extinction level event, because at that point it would be too late. Considering that preventive measures would take time and safety margin, which level of evidence would be sufficient to motivate serious countermeasures?

Re:Tinygrad: Hacked 4090 driver to enable P2P by stale2002 on Friday April 12, 2024 @12:16PM (#40014570)
> Instead of gesturing at flawed analogies

No. Because they are the best way of making my point and they don't allow people to make things much more vague.

The AI doomers trump card argument is to set up vague premise after vague premise, and then slip in the magic somewhere in between all of these vague claims and ignore the specifics.

If there are 10 vague premises, I can't tell you whether it is vague premise number 6, or vague premise number 7 where the magic gets slipped in, because things are so vague that it could be either! But I do know that all the vague premises add up to an extraordinary claim that requires extraordinary evidence.

So for every single argument that you are making, my thought process is going to be as follows.

"Imagine someone told me that pandora's box is real. From Greek mythology. What evidence would I require to believe that?".

Because the claims of diamond nano bot robots and mind controls humans are on the level of extraordinary cliams that are comparable to greek box mythology, even though, yes boxes in real life exist.

Phasing it in such a specific way makes my counter arguments obvious. It doesn't let you say "Well AI technology already exists. Therefore believe me", just like how someone claiming that boxes exist doesn't proves that pandora's box is serious threat.

And it make clear the central point of disagreement. The central point of disagreement, is that jumping from where we are now, to "end of the world" is an extraordinary claim that requires extraordinary evidence.

It is the gap that is large. And just pointing to existing harm with boxes or ai doesn't jump that gap.

> Also, it seems obvious that the standard of evidence that "AI could cause extinction" can't be observing an extinction level event,

Edit I have to bring this one up specifically because it is another extremely bad argument.

We don't have to observe extinction level events. There are 100 steps, way way way before that is even a possibility, that we would observe along the way.

To bring up another example, if there is an observed doomsday device at the end of the universe , there many many years of incremental steps of space travel that we will observe in real life, before we have to worry about a terrorist today building their own universe traveling space ship to fly to that doomsday device.

Same applies to AI. We would observe lots of existing capabilities before we invent world ending magic.

> which level of evidence would be sufficient to motivate serious countermeasures?

Existing real life harm, measurable in the real world, that would justify regulations for that existing harm, regardless of the world ending magic.

We would regulate AI, due to regular normal people problems, long before anyone has to bring up the magic.

Re:Tinygrad: Hacked 4090 driver to enable P2P by cjbprime on Friday April 12, 2024 @11:58AM (#40014373)
What do you think mind control is? Think President Trump but without the self-defeating flaws, with an ability to stick to plans, and most importantly the ability to pay personal attention to each follower to further increase the level of trust and commitment. Not Harry Potter.

People will do what the AI says because it is able to create personal trust relationships with them and they want to help it. (They may not even realize that they are helping an AI rather than a human who cares about them.)

The normal ways that trust is created, not magical ones.

Re:Tinygrad: Hacked 4090 driver to enable P2P by stale2002 on Friday April 12, 2024 @02:46PM (#40016318)
> What do you think mind control is?

The magic technology that is equivalent to the imperious curse from Harry Potter.

> The normal ways that trust is created, not magical ones.

Buildings as a technology are normal. They are constantly getting taller and we have better technology to make them taller.

But, even though buildings are a normal technology, I am not going to worry about buildings getting so tall soon that they hit the sun.

This is the same exact mistake that every single AI doomers makes. What they do is they take something normal, and then they infinitely extrapolate it out to an absurd degree, without admitting that this is an extraordinary claim that requires extraordinary evidence.

The central point of disagreement, that always gets glossed over, is that you can't make a vague claim about how AI is good at stuff, and then do your gigantic leap from here to over there which is "the world ends".

Yes that is the same as comparing these worries to those who worry about buildings hitting the sun or the imperious curse.

Re:Tinygrad: Hacked 4090 driver to enable P2P by FeepingCreature on Friday April 12, 2024 @05:39PM (#40017970)
Less than a month ago: https://arxiv.org/abs/2403.14380 "We found that participants who debated GPT-4 with access to their personal information had 81.7% (p < 0.01; N=820 unique participants) higher odds of increased agreement with their opponents compared to participants who debated humans."

And it's only gonna get better.

Re:Tinygrad: Hacked 4090 driver to enable P2P by stale2002 on Friday April 12, 2024 @09:18PM (#40019497)
Yes, and I am sure that when people do a google search for "Good arguments in favor of X", that they are also sometimes convinced to be more in favor of X.

Perhaps they would be even more convinced by the google search than if a person argued with them about it.

That is still much different from "The AI mind controls people, hacks the nukes, and ends the world".

Its that second part that is the the fantasy land situation that requires extraordinary evidence.

But, this is how conversations about doomsday AI always go. People say "Well isn't AI kinda good at this extremely vague thing Y, sometimes? Imagine if AI was infinitely good at Y! That means that by extrapolation, the world ends!".

And that covers basically every single AI doom argument that anyone ever makes.

Re:Tinygrad: Hacked 4090 driver to enable P2P by FeepingCreature on Saturday April 13, 2024 @08:25AM (#40022698)
If the only evidence for AI doom you will accept is actual AI doom, you are asking for evidence that by definition will be too late.

"Show me the AI mindcontrolling people!" AI mindcontrolling people is what we're trying to avoid seeing.

The trick is, in the world in which AI doom is in the future, what would you expect to see now that is different from the world in which AI doom is not in the future?

Re:Tinygrad: Hacked 4090 driver to enable P2P by stale2002 on Saturday April 13, 2024 @12:31PM (#40024201)
> If the only evidence for AI doom you will accept is actual AI doom

No actually. This is another mistake that the AI doomers make. They pretend like a demand for evidence means that the world has to end first.

Instead, what would be perfectly good evidence, would be evidence of significant incremental harm that requires regulation on its own, independent of any doom argument.

In between "the world literally ends by magic diamond nanobots and mind controlling AI" and "where we are today" would be many many many situations of incrementally escalating and measurable harm that we would see in real life, decades before the world ending magic happens.

We can just treat this like any other technology, and regulate it when it causes real world harm. Because before the world ends by magic, there would be significant real world harm that is similar to any other problem in the world that we handle perfectly well.

Its funny because you committing the exact mistake that I was criticizing in my original post, where you did the absolutely massive jump and hand waved it away.

> what would you expect to see now that is different from the world in which AI doom is not in the future?

What I would expect is for the people who claim to care about AI doom to actually be trying to measure real world harm.

Ironically, I think the people who are coming up with increasingly thin excuses as for why they don't have to find evidence are increasing the likelyhood of such AI doom much more than anyone else because they are abandoning the most effective method of actually convincing the world of the real world damage that AI could cause.

Re:Tinygrad: Hacked 4090 driver to enable P2P by FeepingCreature on Saturday April 13, 2024 @01:16PM (#40024504)
Well, at least if you see escalating measurable harm you'll come around, I'm happy about that. You won't necessarily get the escalating harm even if AI doom is real though, so you should try to discover if it is real even in worlds where hard takeoff is a thing.

> What I would expect is for the people who claim to care about AI doom to actually be trying to measure real world harm.

Why bother? If escalating harm is a thing, everyone will notice. We don't need to bolster that, because ordinary society has it handled.

Re:Tinygrad: Hacked 4090 driver to enable P2P by stale2002 on Saturday April 13, 2024 @05:01PM (#40026087)
> You won't necessarily get the escalating harm even if AI doom is real though

Yes we would. Unless you are one of those people who think that the magic doom nanobots are going to be invented overnight.

My comparisions to someone who is worried about literal magic, from harry potter, is apt.

But at that point, if you are worried about magic showing up instantly, then your position is basically not falsifiable. You can always retreat to some untestable, unfalsifiable magic.

Like there is actually nothing I could say, no evidence I could show to ever convince someone out of that position.

On the other hand, my position is actually fasifiable. There is absolutely all sorts of non world ending evidence that could convince me to think that AI is dangerous.

But nobody on the doomer side seems to care about any of that. Instead they invent positions that seem almost tailor made to avoid being falsifiable or disprovable so that they can continue to believe them despite any evidence to the contrary.

As in, if I were to purposeful invent an idea or philosophy that is impossible to be disproved or convinced out of the "I can't show you evidence because the world will end" position is what I would invent.

> you'll come around,

Do you admit that you won't though? Do you admit that no matter what evidence is shown to you, that you can just retreat and say that the magic could happen at any time?

Or even if this isn't you literally, that someone in your position could dismiss all counter evidence, no matter what, and nobody could convince someone out of that with evidence?

I am not sure how someone could ever possibly engage with you seriously on any of this, if that is your position.

Re:Tinygrad: Hacked 4090 driver to enable P2P by the8472 on Saturday April 13, 2024 @07:04PM (#40027020)
> Like there is actually nothing I could say, no evidence I could show to ever convince someone out of that position.

There is, it is just very hard to obtain. Various formal proofs would do. On upper bounds. On controllability. On scalability of safety techniques.

The manhattan project scientists did check whether they'd ignite the atmosphere before detonating their first prototype. Yes, that was much simpler task. But there's no rule in nature that says proving a system to be safe must be as easy as creating the system. Especially when the concern is that the system adaptive and adversarial.

Recursive self-improvement is a positive feedback loop, like nuclear chain reactions, like virus replication. So if we have an AI that can program then we better make sure that it either cannot sustain such a positive feedback loop or that it remains controllable beyond criticality. Given the complexity of the task it appears unlikely that a simple ten-page paper proving this will show up on arxiv. But if one did that'd be great.

>> You won't necessarily get the escalating harm even if AI doom is real though

> Yes we would.

So what does guarantee a visible catastrophe that won't be attributed to human operators using a non-agentic AI incorrectly? We keep scaling and the systems will be treated as assistants/optimizers and it's always the operators fault. Until we roughly reach human-level on some relevant metrics. And at that point there's a very narrow complexity range from idiot to genius (human brains don't vary by orders of magnitude!). So as far as hardware goes this could be a very narrow range and we could shoot straight from "non-agentic sub-human AI" to "agentic superintelligence" in short timescales once the hardware has that latent capacity. And up until that point it will always have been a human error, lax corporate policies, insufficient filtering of the training set or whatever.

And it's not that it must happen this way. Just that there doesn't seem anything ruling it and similar pathways out.

Re:Tinygrad: Hacked 4090 driver to enable P2P by CamperBob2 on Friday April 12, 2024 @11:14AM (#40013875)
Then it's just a matter of evolution in action.

And while it doesn't take a God to start evolution, it would take a God to stop it.

Re:Tinygrad: Hacked 4090 driver to enable P2P by hollerith on Friday April 12, 2024 @11:16AM (#40013893)
You might be OK with suddenly dying along with all your friends and family, but I am not even if it is "evolution in action".
Re:Tinygrad: Hacked 4090 driver to enable P2P by CamperBob2 on Friday April 12, 2024 @11:22AM (#40013961)
Historically governments haven't needed computers or AI to do that. They've always managed just fine.

Punched cards helped, though, I guess...

Re:Tinygrad: Hacked 4090 driver to enable P2P by FeepingCreature on Friday April 12, 2024 @05:41PM (#40017985)
gestures at the human population graph wordlessly
Re:Tinygrad: Hacked 4090 driver to enable P2P by CamperBob2 on Saturday April 13, 2024 @12:02PM (#40024010)
Agent Smith smiles mirthlessly
Re:Tinygrad: Hacked 4090 driver to enable P2P by the8472 on Friday April 12, 2024 @09:37AM (#40012693)
You say you have not seen any arguments that convince you. Is that just not having seen many arguments or having seen a lot of arguments where each chain contained some fatal flaw? Or something else?
Re:Tinygrad: Hacked 4090 driver to enable P2P by pixl97 on Friday April 12, 2024 @01:17PM (#40015293)
> I see summoning the threat of AGI to pose an existential threat to be on the level with lizard people on the moon.

I mean to every other lifeform on the plant YOU are the AGI existential threat. You, and I mean homosapiens by that, have taken over the planet and have either enslaved and are breeding any other animals for food, or are driving them to extinction. In this light bringing another potential apex predator on to the scene seems rash.

>fall for distractions while their LLM fails at basic math

Correct, if we already had AGI/ASI this discussion would be moot because we'd already be in a world of trouble. The entire point is to slow stuff down before we have a major "oopsie whoopsie we can't take that back" issue with advanced AI, and the best time to set the rules is now.

Re:Tinygrad: Hacked 4090 driver to enable P2P by Aerroon on Friday April 12, 2024 @08:55AM (#40012050)
>If the premise is wrong we won't need it. If society coordinates to not do the dangerous thing we won't need it.

But the idea that this use of force is okay itself increases danger. It creates the situation that actors in the field might realize that at some point they're in danger of this and decide to do a first strike to protect themselves.

I think this is why anti-nuclear policy is not "we will airstrike you if you build nukes" but rather "we will infiltrate your network and try to stop you like that".

Re:Tinygrad: Hacked 4090 driver to enable P2P by wongarsu on Friday April 12, 2024 @10:08AM (#40013091)
> anti-nuclear policy is not "we will airstrike you if you build nukes"

Was that not the official policy during the Bush administration regarding weapons of mass destruction (which covers nuclear weapons in addition to chemical and biological weapons). That was pretty much the official premise of the second Gulf war

Re:Tinygrad: Hacked 4090 driver to enable P2P by FeepingCreature on Friday April 12, 2024 @05:42PM (#40017996)
If Israel couldn't infiltrate Iran's centrifuges, do you think they would just let them have nukes? Of course airstrikes are on the table.
Re:Tinygrad: Hacked 4090 driver to enable P2P by im3w1l on Friday April 12, 2024 @08:57AM (#40012073)
> I'm not seeing the odiousness of the proposal. If bio research gets commodified and easy enough that every kid can build a new airborne virus in their basement we'd need raids on that too.

Either you create even better bio research to neutralize said viruses... or you die trying...

Like if you go with the raid strategy and fail to raid just one terrorist that's it, game over.

Re:Tinygrad: Hacked 4090 driver to enable P2P by the8472 on Friday April 12, 2024 @09:01AM (#40012120)
Those arguments do not transfer well to the AGI topic. You can't create counter-AGI, since that's also an intelligent agent which would be just as dangerous. And chips are more bottlenecked than biologics (... though gene synthesizing machines could be a similar bottleneck and raiding vendors which illegally sell those might be viable in such a scenario).
Re:Tinygrad: Hacked 4090 driver to enable P2P by tomoyoirl on Friday April 12, 2024 @09:01AM (#40012114)
> ... after regulation, court orders and fines have failed

One question for you. In this hypothetical where AGI is truly considered such a grave threat, do you believe the reaction to this threat will be similar to, or substantially gentler than, the reaction to threats we face today like “terrorism” and “drugs”? And, if similar: do you believe suspected drug labs get a court order before the state resorts to a police raid?

> I'm not seeing the odiousness of the proposal.

Well, as regards EliY and airstrikes, I’m more projecting my internal attitude that it is utterly unserious, rather than seriously engaging with whether or not it is odious. But in earnest: if you are proposing a policy that involves air strikes on data centers, you should understand what countries have data centers, and you should understand that this policy risks escalation into a much broader conflict. And if you’re proposing a policy in which conflict between nuclear superpowers is a very plausible outcome — potentially incurring the loss of billions of lives and degradation of the earth’s environment — you really should be able to reason about why people might reasonably think that your proposal is deranged, even if you happen to think it justified by an even greater threat. Failure to understand these concerns will not aid you in overcoming deep skepticism.

Re:Tinygrad: Hacked 4090 driver to enable P2P by the8472 on Friday April 12, 2024 @09:29AM (#40012549)
> In this hypothetical where AGI is truly considered such a grave threat, do you believe the reaction to this threat will be similar to, or substantially gentler than, the reaction to threats we face today like “terrorism” and “drugs”?

"truly considered" does bear a lot of weight here. If policy-makers adopt the viewpoint wholesale, then yes, it follows that policy should also treat this more seriously than "mere" drug trade. Whether that'll actually happen or the response will be inadequate compared to the threat (such as might be said about CO2 emissions) is a subtly different question.

> And, if similar: do you believe suspected drug labs get a court order before the state resorts to a police raid?

Without checking I do assume there'll have been mild cases where for example someone growing cannabis was reported and they got a court summons in the mail or two policemen actually knocking on the door and showing a warrant and giving the person time to call a lawyer rather than an armed, no-knock police raid, yes.

> And if you’re proposing a policy in which conflict between nuclear superpowers is a very plausible outcome — potentially incurring the loss of billions of lives and degradation of the earth’s environment — you really should be able to reason about why people might reasonably think that your proposal is deranged [...]

Said powers already engage in negotiations to limit the existential threats they themselves cause. They have some interest in their continued existence. If we get into a situation where there is another arms race between superpowers and is treated as a conflict rather than something that can be solved by cooperating on disarmament, then yes, obviously international policy will have failed too.

If you start from the position that any serious, globally coordinated regulation - where a few outliers will be brought to heel with sanctions and force - is ultimately doomed then you will of course conclude that anyone proposing regulation is deranged.

But that sounds like hoping that all problems forever can always be solved by locally implemented, partially-enforced, unilateral policies that aren't seen as threats by other players? That defense scales as well or better than offense? Technologies are force-multipliers, as it improves so does the harm that small groups can inflict at scale. If it's not AGI it might be bio-tech or asteroid mining. So eventually we will run into a problem of this type and we need to seriously discuss it without just going by gut reactions.

Re:Tinygrad: Hacked 4090 driver to enable P2P by eek2121 on Friday April 12, 2024 @11:01AM (#40013727)
Just my (probably unpopular) opinion: True AI (what they are now calling AGI) may never exist. Even the AI models of today aren't far removed from the 'chatbots' of yesterday (more like an evolution rather than revolution)...

...for true AI to exist, it would need to be self aware. I don't see that happening in our lifetimes when we don't even know how our own brains work. (There is sooo much we don't know about the human brain.)

AI models today differ only in terms of technology compared to the 'chatbots' of yesterday. None are self aware, and none 'want' to learn because they have no 'wants' or 'needs' outside of their fixed programming. They are little more than glorified auto complete engines.

Don't get me wrong, I'm not insulting the tech. It will have it's place just like any other, but when this bubble pops it's going to ruin lives, and lots of them.

Shoot, maybe I'm wrong and AGI is around the corner, but I will continue to be pessimistic. I am old enough to have gone through numerous bubbles, and they never panned out the way people thought. They also nearly always end in some type of recession.

Re:Tinygrad: Hacked 4090 driver to enable P2P by pixl97 on Friday April 12, 2024 @01:27PM (#40015383)
Why is "Want" even part of your equation.

Bacteria doesn't "want" anything in the sense of active thinking like you do, and yet will render you dead quickly and efficiently while spreading at a near exponential rate. No self awareness necessary.

You keep drawing little circles based on your understanding of the world and going "it's inside this circle, therefore I don't need to worry about it", while ignoring 'semi-smart' optimization systems that can lead to dangerous outcomes.

>I am old enough to have gone through numerous bubbles,

And evidently not old enough to pay attention to the things that did pan out. But hey, those cellphone and that internet thing was just a fad right. We'll go back to land lines at any time now.

Re:Tinygrad: Hacked 4090 driver to enable P2P by Aerroon on Friday April 12, 2024 @08:50AM (#40011995)
I find it baffling that ideas like "govern compute" are even taken seriously. What the hell has happened to the ideals of freedom?! Does the government own us or something?
Re:Tinygrad: Hacked 4090 driver to enable P2P by segfaultbuserr on Friday April 12, 2024 @09:27AM (#40012517)
> I find it baffling that ideas like "govern compute" are even taken seriously.

It's not entirely unreasonable if one truly believes that AI technologies are as dangerous as nuclear weapons. It's a big "if", but it appears that many people across the political spectrum are starting to truly believe it. If one accepts this assumption, then the question simply becomes "how" instead of "why". Depending on one's political position, proposed solutions include academic ones such as finding the ultimate mathematical model that guarantees "AI safety", to Cold War style ones with a level of control similar to Nuclear Non-Proliferation. Even a neo-Luddist solution such as destroying all advanced computing hardware becomes "not unthinkable" (a tech blogger gwern, a well-known personality in AI circles who's generally pro-tech and pro-AI, actually wrote an article years ago on its feasibility through terrorism because he thought it was an interesting hypothetical question).

Re:Tinygrad: Hacked 4090 driver to enable P2P by logicchains on Friday April 12, 2024 @12:32PM (#40014775)
AI is very different from nuclear weapons because a state can't really use nuclear weapons to oppress its own people, but it absolutely can with AI, so for the average human "only the government controls AI" is much more dangerous than "only the government controls nukes".
Re:Tinygrad: Hacked 4090 driver to enable P2P by Filligree on Friday April 12, 2024 @12:47PM (#40014969)
But that makes such rules more likely, not less.
Re:Tinygrad: Hacked 4090 driver to enable P2P by segfaultbuserr on Friday April 12, 2024 @01:00PM (#40015120)
Which is why politicians are going to enforce systematic export regulations to defend the "free world" by stopping “terrorists", and also to stop "rogue states" from using AI to oppress their citizens. /s
Re:Tinygrad: Hacked 4090 driver to enable P2P by LoganDark on Friday April 12, 2024 @01:07PM (#40015191)
I don't think there's any need to be sarcastic about it. That's a very real possibility at this point. For example, the US going insane about how dangerous it is for China to have access to powerful GPU hardware. Why do they hate China so much anyway? Just because Trump was buddy buddy with them for a while?
Re:Tinygrad: Hacked 4090 driver to enable P2P by jprete on Friday April 12, 2024 @10:45AM (#40013520)
If AI is actually capable of fulfilling all the capabilities suggested by people who believe in the singularity, it has far more capacity for harm than nuclear weapons.

I think most people who are strongly pro-AI/pro-acceleration - or, at any rate, not anti-AI - believe that either (A) there is no control problem (B) it will be solved (C) AI won't become independent and agentic (i.e. it won't face evolutionary pressure towards survival) or (D) AI capabilities will hit a ceiling soon (more so than just not becoming agentic).

If you strongly believe, or take as a prior, one of those things, then it makes sense to push the gas as hard as possible.

If you hold the opposite opinions, then it makes perfect sense to push the brakes as hard as possible, which is why "govern compute" can make sense as an idea.

Re:Tinygrad: Hacked 4090 driver to enable P2P by logicchains on Friday April 12, 2024 @12:35PM (#40014822)
>If you hold the opposite opinions, then it makes perfect sense to push the brakes as hard as possible, which is why "govern compute" can make sense as an idea.

The people pushing for "govern compute" are not pushing for "limit everyone's compute", they're pushing for "limit everyone's compute except us". Even if you believe there's going to be AGI, surely it's better to have distributed AGI than to have AGI only in the hands of the elites.

Re:Tinygrad: Hacked 4090 driver to enable P2P by segfaultbuserr on Friday April 12, 2024 @01:10PM (#40015223)
> surely it's better to have distributed AGI than to have AGI only in the hands of the elites.

The argument of doing so is the same as Nuclear Non-Proliferation - because of its great abuse potential, giving the technology to everyone only causes random bombings of cities instead of creating a system with checks and balances.

I do not necessarily agree with it, but I found the reasoning is not groundless.

Re:Tinygrad: Hacked 4090 driver to enable P2P by Aerroon on Saturday April 13, 2024 @11:37AM (#40023810)
But the reason for nuclear non-proliferation is to hold onto power. Abuse potential is a great excuse, but it applies to everyone. Current nuclear states have demonstrated that they are willing to indirectly abuse them (you can't invade Russia, but Russia has no problem invading you as long as you aren't backed up by nukes).
Re:Tinygrad: Hacked 4090 driver to enable P2P by segfaultbuserr on Saturday April 13, 2024 @01:02PM (#40024400)
Both can be true at the same time.

The world's superpowers enforce nuclear non-proliferation mainly because it allows them to keep unfair political and military advantages to themselves. At the same time, one cannot deny that centralized weapon ownership made the use of such weapons more controllable: These nuclear states are powerful enough to establish a somewhat responsible chain of command to avoid their unreasonable or accidental uses, and so far these attempts are still successful. Also, due to the fact that they are "too big to fail", they were forced to hire experts to make detailed analysis on the consequences of nuclear wars, and the resulted MAD doctrine discouraged them from starting such wars.

On the other hand, if the same nuclear technologies are available to everyone, the chance of an unreasonable or accidental nuclear war will be higher. If even resourceful superpowers can barely keep these nuclear weapons under safe political and technical control (as shown by multiple incidents and near-misses during the Cold War [0]), surely a less resourceful state or military in possession of equally destructive weapons will have even more difficulties on controlling their uses.

At least this is how the argument goes (so far, I personally take no position).

Of course, I clearly realized that centralized control is not infallible. Months ago, in a previous thread on OpenAI's refusal on publishing technical details of GPT-4, most people believed that they were using it as an excuse to maintain a monopolistic control. Instead, I argued that perhaps OpenAI truly values the problem of safety right now - but acting responsibly right now is not an indication that they will still act responsibly in the future. There's no guarantee that the safety considerations will eventually be overridden in favor of financial gains.

[0] https://en.wikipedia.org/wiki/Command_and_Control_(book)

Re:Tinygrad: Hacked 4090 driver to enable P2P by Filligree on Friday April 12, 2024 @12:50PM (#40014995)
> surely it's better to have distributed AGI than to have AGI only in the hands of the elites

This is not a given. If your threat model includes "Runaway competition that leads to profit-seekers ignoring safety in a winner-takes-all contest", then the more companies are allowed to play with AI, the worse. Non-monopolies are especially bad.

If your threat model doesn't include that, then the same conclusions sound abhorrent and can be nearly guaranteed to lead to awful consequences.

Neither side is necessarily wrong, and chances are good that the people behind the first set of rules would agree that it'll lead to awful consequences — just not as bad as the alternative.

Re:Tinygrad: Hacked 4090 driver to enable P2P by FeepingCreature on Friday April 12, 2024 @05:37PM (#40017951)
No they really do push for "limit everyone's compute". The people pushing for "limit everyone's compute except us" are allies of convenience that are gonna be inevitably backstabbed.

At any rate, if you have like two corps with lots of compute, and something goes wrong, you only have to EMP two datacenters.

Re:Tinygrad: Hacked 4090 driver to enable P2P by the8472 on Saturday April 13, 2024 @11:09AM (#40023616)
Re:Tinygrad: Hacked 4090 driver to enable P2P by Aerroon on Tuesday April 16, 2024 @06:40AM (#40050437)
Do they also take this position about biological intelligence? Because humans most certainly have an alignment problem too.
Re:Tinygrad: Hacked 4090 driver to enable P2P by the8472 on Wednesday April 17, 2024 @06:27AM (#40062678)
The concerning AGI properties include recursive self-improvement to superhuman capability levels and the ability to mass-deploy copies. Those are not on the horizon when it comes to humans. If hypothetically some human acquired such properties that would be equally concerning.
Re:Tinygrad: Hacked 4090 driver to enable P2P by schlauerfox on Friday April 12, 2024 @07:32PM (#40018715)
This is all just Pascal's wager anyway.
Re:Tinygrad: Hacked 4090 driver to enable P2P by segfaultbuserr on Saturday April 13, 2024 @01:37PM (#40024648)
Surely, there are many realistic ways to abuse this powerful technology, the creation of a self-aware Paperclip Maximizer is not necessarily required.
Re:Tinygrad: Hacked 4090 driver to enable P2P by aftbit on Friday April 12, 2024 @09:45AM (#40012805)
The government sure thinks they own us, because they claim the right to charge us taxes on our private enterprises, draft us to fight in wars that they start, and put us in jail for walking on the wrong part of the street.
Re:Tinygrad: Hacked 4090 driver to enable P2P by andy99 on Friday April 12, 2024 @09:57AM (#40012955)
Taxes, conscription and even pedestrian traffic rules make sense at least to some degree. Restricting "AI" because of what some uninformed politician imagines it to be is in a whole different league.
Re:Tinygrad: Hacked 4090 driver to enable P2P by aftbit on Friday April 12, 2024 @11:06AM (#40013793)
IMO it makes no sense to arrest someone and send them to jail for walking in the street not the sidewalk. Give them a ticket, make them pay a fine, sure, but force them to live in a cage with no access to communications, entertainment, or livelihood? Insane.

Taxes may be necessary, though I can't help but feel that there must be a better way that we have not been smart enough to find yet. Conscription... is a fact of war, where many evil things must be done in the name of survival.

Regardless of our views on the ethical validity or societal value of these laws, I think their very existence shows that the government believes it "owns" us in the sense that it can unilaterally deprive us of life, liberty, and property without our consent. I don't see how this is really different in kind from depriving us of the right to make and own certain kinds of hardware. They regulated crypto products as munitions (at least for export) back in the 90s. Perhaps they will do the same for AI products in the future. "Common sense" computer control.

Re:Tinygrad: Hacked 4090 driver to enable P2P by zoklet-enjoyer on Friday April 12, 2024 @11:55AM (#40014325)
The US draft in the Vietnam war had nothing to do with the survival of the US
Re:Tinygrad: Hacked 4090 driver to enable P2P by aftbit on Friday April 12, 2024 @01:12PM (#40015234)
I feel a bit like everyone is missing the point here. Regardless of whether law A or law B is ethical and reasonable, the very existence of laws and the state monopoly on violence suggests a privileged position of power. I am attempting to engage with the word "own" from the parent post. I believe the government does in fact believe it "owns" the people in a non-trivial way.
Re:Tinygrad: Hacked 4090 driver to enable P2P by pixl97 on Friday April 12, 2024 @01:10PM (#40015219)
Are you allowed to store as many dangerous chemicals at your house as you like? No. I guess the government owns you or something.
Re:Tinygrad: Hacked 4090 driver to enable P2P by int_19h on Friday April 12, 2024 @09:50PM (#40019695)
It's the same thing as always happens with freedom.

"But foreign propagandists ..."

"But extremists ..."

"But terrorists ..."

"But child abusers ..."

Re:Tinygrad: Hacked 4090 driver to enable P2P by snakeyjake on Friday April 12, 2024 @09:20AM (#40012416)
I love the HN dystopian fantasies.

They're simply adorable.

They're like how jesusfreaks are constantly predicting the end times, with less mass suicide.

Re:Tinygrad: Hacked 4090 driver to enable P2P by erikbye on Friday April 12, 2024 @09:35AM (#40012647)
We already have export restrictions on cryptography. Of course there will be AI regulations.
Re:Tinygrad: Hacked 4090 driver to enable P2P by snakeyjake on Friday April 12, 2024 @11:52AM (#40014287)
You need to abandon your apocalyptic worldview keep up with the times my friend.

Encryption export controls have been systematically dismantled to the point that they're practically non-existent, especially over the last three years.

Pretty much the only encryption products you need permission to export are those specifically designed for integration into military communications networks, like Digital Subscriber Voice Terminals or Secure Terminal Equipment phones, everything else you file a form.

Many things have changed since the days when Windows 2000 shipped with a floppy disk containing strong encryption for use in certain markets.

https://archive.org/details/highencryptionfloppydisk

Re:Tinygrad: Hacked 4090 driver to enable P2P by erikbye on Friday April 12, 2024 @12:38PM (#40014856)
Are you on drugs or is your reading comprehension that poor?

1) I did not state a world view; I simply noted that restrictions for software do exist, and will for AI, as well. As the link from the other commenter show, they do in fact already exist.

2) Look up the definition of "apocalyptic", software restrictions are not within its bounds.

3) How the restrictions are enforced were not a subject in my comment.

4) We're not pals, so you can drop the "friend", just stick to the subject at hand.

Re:Tinygrad: Hacked 4090 driver to enable P2P by snakeyjake on Saturday April 13, 2024 @02:40AM (#40021054)
I'm high on life, old chum!
Re:Tinygrad: Hacked 4090 driver to enable P2P by erikbye on Monday April 15, 2024 @02:19PM (#40043920)
You enjoy, then :)
Re:Tinygrad: Hacked 4090 driver to enable P2P by Jerrrry on Friday April 12, 2024 @09:59AM (#40012990)
>Of course there will be AI regulations.

Are. As I and others have predicted, the executive order was passed defining a hard limit on the processing/compute power allowed without first 'checkin in' with the Letter boys.

https://www.whitehouse.gov/briefing-room/presidential-action...

Re:Tinygrad: Hacked 4090 driver to enable P2P by int_19h on Friday April 12, 2024 @10:05PM (#40019805)
I wonder if we can squeeze a 20b parameter model into a book somehow...
Re:Tinygrad: Hacked 4090 driver to enable P2P by Kuinox on Friday April 12, 2024 @12:24PM (#40014674)
If it could be another acronym than the renowned french Atomic Energy Commission, the CEA.
Re:Tinygrad: Hacked 4090 driver to enable P2P by baobun on Friday April 12, 2024 @03:41PM (#40016851)
Wow, that was a ride. Really pushing the Overton window.

"Regulating access to compute rather than data" - they're really spelling out their defection in the war on access to general computation.

Re:Tinygrad: Hacked 4090 driver to enable P2P by FeepingCreature on Friday April 12, 2024 @05:35PM (#40017936)
I mean yeah they (and I) think if you have too much access to general computation you can destroy the world.

This isn't a "defection", because this was never something they cared about preserving at the risk of humanity. They were never in whatever alliance you're imagining.