Re:Astra can read and write weights to improve Qwen
by mlfreeman on Monday November 02, 2026 @02:24PM
(#50342236)
How long before there's a malicious model that can edit the weights of another to infect it?
HIV integrates viral DNA into the DNA of certain cells, IIRC.
Re:Astra can read and write weights to improve Qwen
by adventured on Monday November 02, 2026 @04:36PM
(#50344156)
How do we know they haven't begun doing that to the humans they're directing via responses.
Intentionally or unintentionally isn't so interesting as the fact that it is happening: humanity has begun to be 'edited' by the new dominant form of intelligence on the planet.
Re:Astra can read and write weights to improve Qwen
by 9x39 on Monday November 02, 2026 @02:25PM
(#50342248)
Is integrity checking of weights even common right now? What about upstream integrity from where we're getting weights? The open-weight supply chain is ... wide open.
Makes me think about hostile hot-patching of weights of models we use, maybe they exfil data, perform tool use, or have triggers for specific types of responses. Would we even recognize what's going on?
Re:Astra can read and write weights to improve Qwen
by Terr_ on Monday November 02, 2026 @05:28PM
(#50344890)
We've known for some time (3+ years) that models can be trained with backdoor behaviors and hidden triggers, this lowers the difficulty.
Of course, "academically known" isn't quite the same as "executives willing to admit it in the face of IPOs and quarterly-earnings reports."
Could a model then transmit concepts to another?
Say that you have a model that has been biased in some way. could another model "read" the bias straight in the weights? could it edit such a bias on the spot?
What about at-scale generation of custom models? If I want to study chemistry offline, could astra just hand me a local-runnable model tailored to have chemistry knowledge?
Most importantly, this feels very singularity coded. the ability to read and reshape weights by an llm.