News for hackers. Stuff that matters.
Re:Models can now transmit viruses in weight-space by aatd86 on Friday November 13, 2026 @02:21PM (#50488426)
All the more reasons for openweights and model hashing.
Re:Models can now transmit viruses in weight-space by ShinyLeftPad on Friday November 13, 2026 @02:24PM (#50488467)
And it's going to help spot the "virus" how?
Re:Models can now transmit viruses in weight-space by garciasn on Friday November 13, 2026 @02:29PM (#50488517)
I recently spoke with a researcher who was working on a statistical method for model deception detection. They trained Qwen to behave in a nefarious way when prompted with a specific phrase and the model then called a tool in a non-transparent way to the end user, reporting back a non-tool call response in a completely benign manner.

Apparently, based on whatever research or paper they were using as their foundation for this statistical model, they could spot the 'virus'. Perhaps it would be in a similar fashion?

Re:Models can now transmit viruses in weight-space by bee_rider on Friday November 13, 2026 @03:57PM (#50489552)
Did they (somehow, this seems extremely difficult) notice a pattern in the weights that indicated tampering, or just scan for the known manipulated weights? The former seems really useful and very difficult, while the latter seems less of both.
Re:Models can now transmit viruses in weight-space by garciasn on Friday November 13, 2026 @05:13PM (#50490730)
I didn't ask and they didn't directly say; but, from what I took away from the conversation it was direct observation of the weights.
Re:Models can now transmit viruses in weight-space by kirth_gersen on Friday November 13, 2026 @02:39PM (#50488648)
more eyes would seem to raise the odds of spotting malicious tampering. or at least create the opportunity for people to look for malicious tampering. if the weights are closed then even looking is impossible...
Re:Models can now transmit viruses in weight-space by lp92 on Friday November 13, 2026 @02:42PM (#50488679)
Your logic is exactly what the Linux people were saying. But as we saw due to AI more and more bugs and exploits are being found, some dating back years of releases.

Open weights won't prevent a model from being poisoned because no human understands how the weights actually impact the model.

Re:Models can now transmit viruses in weight-space by prophesi on Friday November 13, 2026 @02:54PM (#50488797)
> But as we saw due to AI more and more bugs and exploits are being found, some dating back years of releases.

That's by design though. It's just that there's now an exponential increase in the amount of "eyes", in the form of AI agents, looking at the FOSS code to find and fix the bugs.

Re:Models can now transmit viruses in weight-space by bigbadfeline on Friday November 13, 2026 @03:02PM (#50488880)
> Your logic is exactly what the Linux people were saying.

There's nothing wrong with that logic.

> But as we saw due to AI more and more bugs and exploits are being found, some dating back years of releases.

Well, AI helps fixing bugs too and it can also write bug free code with minimal additional effort, so open source not only retains its advantages in the presence of AI, it multiplies them. After the initial (and artificial) hype and bug-fixing efforts things are going to settle down quite nicely.

Closed source, on the other hand, doesn't have the advantage of multiple eyes and minds finding and fixing bugs and the unavailability of the source isn't an advantage either - AI can decompile and understand binary too, so it can find bugs there without access to the sources as well.

Re:Models can now transmit viruses in weight-space by badsectoracula on Friday November 13, 2026 @08:41PM (#50492682)
> But as we saw due to AI more and more bugs and exploits are being found, some dating back years of releases.

The source code being available doesn't mean there wont be any bugs, it only means that it'll be possible to find and fix the bugs without waiting whoever gatekeeps access to that code to do it for you.

Similarly, the weights being open, means that 3rd parties can potentially locate and fix issues with them without waiting for Qwen, ZhipuAI, Mistral, or whoever else to do it (who at that point might as well have moved on).

And yes, open weights wont prevent a model from being poisoned, just like open source doesn't prevent software from becoming buggy or exploitable. Where it helps is for 3rd parties being able to find, fix and release the fixes without depending on the original creators.

Re:Models can now transmit viruses in weight-space by bigbadfeline on Friday November 13, 2026 @02:51PM (#50488767)
Something similar to abliteration would work, after all abliteration removes the alignment virus, why not the others? Fine-tuning can be used too, there are many options as long as the weights are available.

On the other hand, nothing can be done to fix closed models, nobody knows how they work, how they select the targets for their attacks or the nature of those attacks - even conventional alignment can introduce dangerous cognitive viruses.

The only way to be sure is to have open weights models with fully open abliteration and fine-tuning - both the process and data.