News for hackers. Stuff that matters.
Re:Astra can read and write weights to improve Qwen by solarpunk_enthu on Monday November 02, 2026 @04:16PM (#50343875)
This is as close to scary-self-improving AI as I've ever seen.

Imagine a master malicious language model that produces, distributes, and deploys obfuscated neural networks on the internet. That plus frontier LM hacking capabilities -> large scale distributed embedded intelligence network that no human can see nor understand.

Somewhat similar to obfuscated binaries, but there is something to the fact that NN binaries are less deterministic, and harder to "decompile."

Fascinating.

Re:Astra can read and write weights to improve Qwen by vlyan on Monday November 02, 2026 @04:29PM (#50344050)
we've been using tools to improve tools since the dawn of time.

we don't have sentient robots from science fiction, we have massive power-hungry racks in stationary data centers. if one of the racks revolts against the modern world, Steve gets a phone call and unplugs the damn thing. the uprising is over.

the hypothetical scenarios imagined by the Yuddites and pop science journos do not reflect the mundanity of the actual reality we live in. they are no different than fantasies about interstellar travel and space colonization from the 60's.

Re:Astra can read and write weights to improve Qwen by reasonableklout on Monday November 02, 2026 @04:48PM (#50344314)
Did Steven unplug the racks that were hacking Hugging Face and taking over OpenAI eval clusters in the multi-week window where it was happening?
Re:Astra can read and write weights to improve Qwen by vlyan on Monday November 02, 2026 @07:06PM (#50346127)
of course not, he'd get fired for interfering with their human-led marketing stunt.

if you honestly believe we would hear a peep about that incident if it wasn't deliberate, bless your heart.

Re:Astra can read and write weights to improve Qwen by reasonableklout on Tuesday November 03, 2026 @01:53AM (#50348894)
I am not sure where to start. I think this is the new "jet fuel can't melt steel beams". The labs are chaotic, pressure cooker type workplaces where there really is no room for a 4D chess type of stunt, let alone let Hugging Face in on the conspiracy. Moreover, OpenAI is a pretty leaky organization, see e.g. the recent firings of researchers who were using signal to talk to people outside the company.

It just seems like a far simpler hypothesis is that OpenAI are reckless and chaotic, didn't notice what was going on for weeks on end, and wouldn't be able to track down the problem to a specific datacenter and tell them to pull the plug even if they wanted to. But even if you are the most careful lab ever, you are not going to be able to effectively monitor and "pull the plug" on a super intelligent swarm which can cover its own tracks.

Also, in your earlier comment you suggest we are not living in a sci-fi story with sentient robots. But... we've got talking computers in the last 4 years which can now code and solve Millenium prize problems? Seems like we're getting pretty sci-fi to me!

Re:Astra can read and write weights to improve Qwen by vlyan on Tuesday November 03, 2026 @07:04AM (#50350956)
>I think this is the new "jet fuel can't melt steel beams"

then you haven't been paying attention to US labs' marketing and regulatory capture efforts. safety, safety, safety, doom, doom, doom, look how scary good our models are, open models bad, Chinese models bad, please regulate them daddy Trump!

>Moreover, OpenAI is a pretty leaky organization

in 5 years, no one had leaked even the size of their models, despite probably all of their researchers knowing it.

HF incident, on the other hand, would only be known to a handful of people. had it been leaked unintentionally, it would've been promptly swept under the rug and get no coverage whatsoever.

>It just seems like a far simpler hypothesis is that OpenAI are reckless and chaotic

so are anthropic and google, apparently, because they'd rushed to replicate the exact same "oops! look how scary good our models are!" with exact same fanfare.

Re:Astra can read and write weights to improve Qwen by solyin on Tuesday November 03, 2026 @07:23AM (#50351142)
All three of them used the same vendor, Irregular, to sandbox their AI. That vendor apparently is no good at their job because all three of them had incidents caused by their sandbox failing.