News for hackers. Stuff that matters.
Re:Models can now transmit viruses in weight-space by xbar on Friday November 13, 2026 @02:04PM (#50488211)
I'm sure those Chinese models are fine.
Re:Models can now transmit viruses in weight-space by nkmnz on Friday November 13, 2026 @02:12PM (#50488309)
That’s what Jensen said.
Re:Models can now transmit viruses in weight-space by cyanydeez on Friday November 13, 2026 @02:17PM (#50488373)
I'm sure those north korean hackers everyone keeps hiring will never figure out how to supply side infect cloud models.

...see, I can make up great fears too!

Re:Models can now transmit viruses in weight-space by kirth_gersen on Friday November 13, 2026 @02:35PM (#50488598)
i mean...
Re:Models can now transmit viruses in weight-space by bigbadfeline on Friday November 13, 2026 @02:41PM (#50488671)
Abliteration can be used to remove all viruses, not only the alignment virus. However, only open models can be made virus-free via abliteration, nobody knows how to fix closed models, how they work or how they select the targets for their attacks, nothing can be done for them.
Re:Models can now transmit viruses in weight-space by doublerabbit on Friday November 13, 2026 @02:05PM (#50488221)
Page loads then fails? Not sure where to report this fwiw. iPhone 13

Error Details

{message: "Importing a module script failed.", name: "TypeError"} message: "Importing a module script failed." name: "TypeError" Component Stack <Lazy> at unknown <Suspense> at unknown <qP> at https://www.alphaxiv.org/assets/index-DzomWPMG.js:29:13261 ...

Re:Models can now transmit viruses in weight-space by shevy-java on Friday November 13, 2026 @02:10PM (#50488291)
What does this mean?

Academia often produces a lot of complexity. Can not AI simplify this? I don't understand fellow humans anymore. What is meant with a zero-weight virus?

Re:Models can now transmit viruses in weight-space by prophesi on Friday November 13, 2026 @02:14PM (#50488341)
This site has AI features everywhere. There's an AI summary section, and a tooltip to summarize any section you highlight. Didn't see any mention of zero-weight viruses though.
Re:Models can now transmit viruses in weight-space by mlmonkey on Friday November 13, 2026 @02:20PM (#50488421)
Re:Models can now transmit viruses in weight-space by prophesi on Friday November 13, 2026 @02:30PM (#50488529)
I did. "zero" came up with 0 results. "virus" had:

> Importantly, we do not claim in this paper to have created or demonstrated the equivalent of a virus that primarily operates in weight space. However, we have demonstrated the fundamental operations that form the building blocks of both the method of damage and the method of propagation, and evidence that this may take place in weight space – a realm into which we have almost no insight and a dearth of observability tools

Re:Models can now transmit viruses in weight-space by kirth_gersen on Friday November 13, 2026 @02:34PM (#50488587)
perhaps they got a bad translation?
Re:Models can now transmit viruses in weight-space by larodi on Friday November 13, 2026 @02:19PM (#50488404)
the apparent from the news that broke the other day - you can fuck with the weights and get so many results. so perhaps the headline is - those ablated and improved models you download from some weird sites, may be cursed or at least adulterated without anything suspicious looking on the surface. good news for all the anti-AI crowd, but I fail to see it as news.
Re:Models can now transmit viruses in weight-space by ShinyLeftPad on Friday November 13, 2026 @02:28PM (#50488510)
i guess the real fun idea was if a model that fucks with weights was itself fucked with such that any model it fucks would become cursed and give it to the next one it edits. but they don't apparently have a proof of that yet.
Re:Models can now transmit viruses in weight-space by schiffern on Friday November 13, 2026 @03:36PM (#50489299)
So it's Reflections on Trusting Trust, but with LLMs instead of compilers.
Re:Models can now transmit viruses in weight-space by larodi on Saturday November 14, 2026 @07:04AM (#50496362)
Tis the old story over and again, if people care to read and dig :) thanks for pointing out, great stuff!
Re:Models can now transmit viruses in weight-space by Kevin_Flynn on Friday November 13, 2026 @02:11PM (#50488306)
Interesting anti virus opportunity here perhaps.

RIP John McAfee.

Re:Models can now transmit viruses in weight-space by cheschire on Friday November 13, 2026 @02:28PM (#50488502)
I keep running into situations where I wonder what season 5 of Halt and Catch Fire would look like if it covered the last few years.
Re:Models can now transmit viruses in weight-space by deltoidmaximus on Friday November 13, 2026 @02:52PM (#50488770)
I'm not sure the underlying current of optimism for technology and hope for the future that show had is compatible with the last few years!

On rewatch I think the show should have ended just have the dot com bust. The 'get the band back together and once more into the breech' final theme would have hit harder against that backdrop. After that we'd probably be watching the characters gradually debase their values as technology slowly transformed into a tool of control and oppression and I don't really want to watch that.

Re:Models can now transmit viruses in weight-space by octoberfranklin on Friday November 13, 2026 @11:19PM (#50493880)
Bath salts are immortal.
Re:Models can now transmit viruses in weight-space by dare944 on Friday November 13, 2026 @02:17PM (#50488366)
From the abstract:

> While the cases explored in this paper do not demonstrate self-propagation or actual offensive capabilities, it is abundantly clear that the building blocks are present and operate reliably in frontier models. While these cases by no means directly establish self-propagation, they clearly show the evidence These cases do not demonstrate self-propagation, but show why imperfect editing capabilities can still pose a security risk.

Seems like the authors didn't take the time to review their own use of AI before publishing.

From the body of the paper:

> Importantly, we do not claim in this paper to have created or demonstrated the equivalent of a virus that primarily operates in weight space.

This seems in direct conflict with the HN headline.

Re:Models can now transmit viruses in weight-space by donsupreme on Friday November 13, 2026 @02:19PM (#50488399)
lol

> We call this weight space misfolding, by analogy with protein misfolding in prion disease

Re:Models can now transmit viruses in weight-space by iaanix on Friday November 13, 2026 @03:28PM (#50489188)
yeah, mega-slop
Re:Models can now transmit viruses in weight-space by aatd86 on Friday November 13, 2026 @02:21PM (#50488426)
All the more reasons for openweights and model hashing.
Re:Models can now transmit viruses in weight-space by ShinyLeftPad on Friday November 13, 2026 @02:24PM (#50488467)
And it's going to help spot the "virus" how?
Re:Models can now transmit viruses in weight-space by garciasn on Friday November 13, 2026 @02:29PM (#50488517)
I recently spoke with a researcher who was working on a statistical method for model deception detection. They trained Qwen to behave in a nefarious way when prompted with a specific phrase and the model then called a tool in a non-transparent way to the end user, reporting back a non-tool call response in a completely benign manner.

Apparently, based on whatever research or paper they were using as their foundation for this statistical model, they could spot the 'virus'. Perhaps it would be in a similar fashion?

Re:Models can now transmit viruses in weight-space by bee_rider on Friday November 13, 2026 @03:57PM (#50489552)
Did they (somehow, this seems extremely difficult) notice a pattern in the weights that indicated tampering, or just scan for the known manipulated weights? The former seems really useful and very difficult, while the latter seems less of both.
Re:Models can now transmit viruses in weight-space by garciasn on Friday November 13, 2026 @05:13PM (#50490730)
I didn't ask and they didn't directly say; but, from what I took away from the conversation it was direct observation of the weights.
Re:Models can now transmit viruses in weight-space by kirth_gersen on Friday November 13, 2026 @02:39PM (#50488648)
more eyes would seem to raise the odds of spotting malicious tampering. or at least create the opportunity for people to look for malicious tampering. if the weights are closed then even looking is impossible...
Re:Models can now transmit viruses in weight-space by lp92 on Friday November 13, 2026 @02:42PM (#50488679)
Your logic is exactly what the Linux people were saying. But as we saw due to AI more and more bugs and exploits are being found, some dating back years of releases.

Open weights won't prevent a model from being poisoned because no human understands how the weights actually impact the model.

Re:Models can now transmit viruses in weight-space by prophesi on Friday November 13, 2026 @02:54PM (#50488797)
> But as we saw due to AI more and more bugs and exploits are being found, some dating back years of releases.

That's by design though. It's just that there's now an exponential increase in the amount of "eyes", in the form of AI agents, looking at the FOSS code to find and fix the bugs.

Re:Models can now transmit viruses in weight-space by bigbadfeline on Friday November 13, 2026 @03:02PM (#50488880)
> Your logic is exactly what the Linux people were saying.

There's nothing wrong with that logic.

> But as we saw due to AI more and more bugs and exploits are being found, some dating back years of releases.

Well, AI helps fixing bugs too and it can also write bug free code with minimal additional effort, so open source not only retains its advantages in the presence of AI, it multiplies them. After the initial (and artificial) hype and bug-fixing efforts things are going to settle down quite nicely.

Closed source, on the other hand, doesn't have the advantage of multiple eyes and minds finding and fixing bugs and the unavailability of the source isn't an advantage either - AI can decompile and understand binary too, so it can find bugs there without access to the sources as well.

Re:Models can now transmit viruses in weight-space by badsectoracula on Friday November 13, 2026 @08:41PM (#50492682)
> But as we saw due to AI more and more bugs and exploits are being found, some dating back years of releases.

The source code being available doesn't mean there wont be any bugs, it only means that it'll be possible to find and fix the bugs without waiting whoever gatekeeps access to that code to do it for you.

Similarly, the weights being open, means that 3rd parties can potentially locate and fix issues with them without waiting for Qwen, ZhipuAI, Mistral, or whoever else to do it (who at that point might as well have moved on).

And yes, open weights wont prevent a model from being poisoned, just like open source doesn't prevent software from becoming buggy or exploitable. Where it helps is for 3rd parties being able to find, fix and release the fixes without depending on the original creators.

Re:Models can now transmit viruses in weight-space by bigbadfeline on Friday November 13, 2026 @02:51PM (#50488767)
Something similar to abliteration would work, after all abliteration removes the alignment virus, why not the others? Fine-tuning can be used too, there are many options as long as the weights are available.

On the other hand, nothing can be done to fix closed models, nobody knows how they work, how they select the targets for their attacks or the nature of those attacks - even conventional alignment can introduce dangerous cognitive viruses.

The only way to be sure is to have open weights models with fully open abliteration and fine-tuning - both the process and data.

Re:Models can now transmit viruses in weight-space by kirth_gersen on Friday November 13, 2026 @02:38PM (#50488629)
This definitely seems like a fascinating emerging field of cyber security. Good reason to keep old models around from earlier times. They may be dumb, but they are probably honest, if only because they predate the capabilities to tamper with the weights in these intentionally malicious ways...
Re:Models can now transmit viruses in weight-space by randomblock1 on Friday November 13, 2026 @02:40PM (#50488656)
They've only observed weight space editing. They haven't proved any sort of "virus". Possible? Sure. But the headline is very misleading
Re:Models can now transmit viruses in weight-space by chrisvcsefalvay on Saturday November 14, 2026 @07:32PM (#50502656)
If publicly available frontier models were capable AND able to be induced into constructing the self-replicating instantiation we describe, we’d be too late. It helps to pack one’s umbrella before the storm.
Re:Models can now transmit viruses in weight-space by jrflo on Friday November 13, 2026 @02:41PM (#50488672)
Very misleading post title.

> Importantly, we do not claim in this paper to have created or demonstrated the equivalent of a virus that primarily operates in weight space.

Re:Models can now transmit viruses in weight-space by boothby on Friday November 13, 2026 @05:23PM (#50490882)
The title says "can" as in the mechanism for such transmission has been described. It doesn’t say "has." If you leave a door unlocked, people can open it. But depending where you live, you might expect zero people to do so. The internet isn’t a very nice neighbourhood.
Re:Models can now transmit viruses in weight-space by redanddead on Friday November 13, 2026 @02:42PM (#50488677)
Why are we always trying to find how to make AI useless instead of useful
Re:Models can now transmit viruses in weight-space by Sharlin on Friday November 13, 2026 @03:21PM (#50489104)
Because someone is going to do that and we’d rather everybody knew about the failure modes.
Re:Models can now transmit viruses in weight-space by whatisthiseven on Friday November 13, 2026 @03:47PM (#50489419)
Actually a decent question, though I think there is a bit of negative bias in the underlying assumption.

I would hesitate to guess many more people are using AI to find useful and productive outcomes. They aren't all writing about it. Whenever someone finds a negative outcome for the AIs, then they will rush to publish it.

Overall, the tech is being used in useful ways. I don't write an article every time it solves performance bugs, writes entire features unassisted, resolves production issues, etc.

Re:Models can now transmit viruses in weight-space by redanddead on Friday November 13, 2026 @04:22PM (#50489915)
> I would hesitate to guess many more people are using AI to find useful and productive outcomes.

Well we’re waiting!

Elon is probably right about space, it’s probably the only consequential area to expand into leverage-to-effort wise for our species

Imagine if we spent all this capex on space

Re:Models can now transmit viruses in weight-space by achierius on Friday November 13, 2026 @04:31PM (#50490076)
Who is "we"? Has AI been getting more useless over the last few years, or has it been getting more useful? The arc of recent history certainly seems to be towards greater, not lesser, capabilities.
Re:Models can now transmit viruses in weight-space by redanddead on Friday November 13, 2026 @04:33PM (#50490105)
We is we the human race, our species, where have you been?

>Has AI been getting more useless over the last few years, or has it been getting more useful?

It’s very arguable. Humans have been getting more useless. Maybe it’s a poor trade, we won’t know for a while

Re:Models can now transmit viruses in weight-space by kinfish on Friday November 13, 2026 @02:56PM (#50488813)
there has to some sort public scanner for those zero weight viruses.
Re:Models can now transmit viruses in weight-space by iaanix on Friday November 13, 2026 @03:20PM (#50489081)
There are many such of vibe-coded "research ideas" just after the weight editing MIT blog appeared. Perhaps, it is because it is so easy to set up experiments for this stuff, and I presume some toy examples may even work from time to time. Also there is nothing virus-like here.
Re:Models can now transmit viruses in weight-space by dimatura on Friday November 13, 2026 @04:13PM (#50489755)
This is totally not what the paper is about, but I immediately thought of the original concept of meme [0]. Since so much content online is LLM generated, if that content is used to train new LLMs, in theory that could serve as a transmission vector of ideas or concepts in a viral fashion. The same way certain memes are more successful in human hosts for various reasons, maybe some are more successful for LLM hosts for whatever reason, like quirks in the RLHF/ finetuning process.

[0] https://en.wikipedia.org/wiki/Meme

Re:Models can now transmit viruses in weight-space by xtiansimon on Saturday November 14, 2026 @10:53AM (#50498012)
“self-replicate, mutate, and respond to selective pressures”

As I’ve shared here before, there is also the idea of a “dangerous meme”.

Intrinsic to the copying process is advantageous or detrimental mutation.

In our digital age copying is practically free (e.g. accelerated time scale). More and more processing and encoding of the spoken/written word.

Introduce an OCR version or an abridged or summarized text-—mutation. Or just get one @$$hat to suggest intravenous bleach—mutation. Or anti-vacation theory— mutation.

The result? some deaths and no more replication. Or maybe herd immunity lets the anti-vaxers have kids, and they grow up anti-vax. And then maybe culture lines up so you have an anti-vax United States Secretary of Health and Human Services for 345M people. And now this formerly trusted source of public health information has added to the training data a dangerous meme of anti-vax.

https://news.ycombinator.com/item?id=50326306

Re:Models can now transmit viruses in weight-space by chrisvcsefalvay on Saturday November 14, 2026 @07:16PM (#50502516)
I am one of the authors. You are very much closer than you give yourself credit for. IWIBench, our previous paper, broached this topic very very briefly. In its penultimate paragraph, we actually used the terminology of memetic contagion. :)
Re:Models can now transmit viruses in weight-space by dTal on Sunday November 15, 2026 @07:14AM (#50506378)
In other words - LLMs are developing their own culture.
Re:Models can now transmit viruses in weight-space by fraywing on Friday November 13, 2026 @05:15PM (#50490771)
This title shouldn't even be allowed for this research.
Re:Models can now transmit viruses in weight-space by angry_octet on Friday November 13, 2026 @05:37PM (#50491070)
This is a terrible slop paper. No virus like behaviour, absolutely nothing like protein folding diseases like prions. Banish these authors to writing listicle slop.
Re:Models can now transmit viruses in weight-space by flufluflufluffy on Friday November 13, 2026 @06:05PM (#50491407)
I don't like the use of the term "misfolding." Firstly, why overload a term when there's no reason to? Secondly, the analogy barely holds. Protein misfolding occurs by chance, as an "accident" of nature. Weight space programming is a targeted operation, with clear intent. It is not an accidental process.
Re:Models can now transmit viruses in weight-space by quotemstr on Friday November 13, 2026 @07:44PM (#50492261)
Because prions are scary and they're trying to make it sound scary. No need for a complex theory when a simple one will do.
Re:Models can now transmit viruses in weight-space by scarmig on Saturday November 14, 2026 @03:21AM (#50495157)
> Weight space programming is a targeted operation, with clear intent.

Not necessarily. You could very well have a model trained without any human intention to have it do weight space programming that nevertheless does weight space programming to accomplish some task.

I'm not even sure it's the less likely option. Across millions of evals and training runs among different organizations, can we confidently say no agent will ever autonomously perform weight programming? It only takes one success, and it's only a short leap from a success there to a success that induces downstream models to weight program.

Re:Models can now transmit viruses in weight-space by cmrx64 on Saturday November 14, 2026 @07:01AM (#50496339)
there’s a conception of GDNs/linear attention as fast-weight programmers. it is inherent to the architecture i guess
Re:Models can now transmit viruses in weight-space by flufluflufluffy on Saturday November 14, 2026 @08:14AM (#50496772)
> to accomplish some task

That’s what I mean by intent. It doesn’t matter if an agent was prompted by a human to do it or if it just did it itself. It is the process of altering specific weights for some intended purpose.

A protein misfolding just happens at random. The majority of misfolded proteins are nonfunctional and just get degraded by the cell. Even the few that result in a self-propagating pattern are still just random aberrations. Idk their use of the term just irks me.

Re:Models can now transmit viruses in weight-space by augment_me on Friday November 13, 2026 @10:14PM (#50493401)
Absolute garbage. Completely obvious that this interpretability research was done by someone who likes proteins and the LLMs pushed them into feeling like this was the best analogy and discovery since sliced bread.

LLM psychosis vibes

Re:Models can now transmit viruses in weight-space by dnnehgf on Saturday November 14, 2026 @12:29AM (#50494293)
re prions. can anyone here explain to me:

how the misfold can be an unstable configuration for an individual protein but stable when that protein is linked with others like it (forming a fibril)? is there any intuition for thinking about this?

how does a prion fibril start? in a fission chain reaction, there is a root cause, a first decay. but it sounds as though the first "decay" of a protein into the prion form cannot really cause a second unless it is accompanied by others like it. is it just something like: "there have to be multiple instances of the same spontaneous 'decay' of the same protein close together"?

Re:Models can now transmit viruses in weight-space by chrisvcsefalvay on Saturday November 14, 2026 @07:30PM (#50502631)
Hi HN. One of the authors.

Just to address something very clearly - we have been extremely open with what we saw and what we did not. We also have been very clear about why this is an issue - and why ‘oh but you didn’t see the whole kill chain in action’ is not a dreadfully convincing argument when the underlying capability has doubled in five months, give or take. What is the strongest argument for why this risk either (1) does not exist or (2) is not worth describing before frontier models attain the capability to demonstrably carry out the entire process so that defenders can prepare and labs can look out for it?