News for hackers. Stuff that matters.

A comment on Astra can read and write weights to improve Qwen.

Re:Astra can read and write weights to improve Qwen by mlmonkey on Monday November 02, 2026 @09:22PM (#50347180)
The question I've been pondering is: does this work only with bigger models modifying smaller ones? Or can a smaller model tweak and improve a bigger one? I'm guessing a whole bunch of GPUs are overheating in some datacenters out there trying to answer this very question.

My hunch is: one needs a model of a similar or bigger capacity to perform surgery on a smaller model.

[ Parent | on HN ]
Re:Astra can read and write weights to improve Qwen by jdub on Monday November 02, 2026 @10:05PM (#50347506)
There's no need to believe the direction is magical, or analogous to "smarter" teaching "stupider".

A larger model may be more capable at processing/interpreting neural network weights. A smaller model may be easier to process/interpret.

That doesn't preclude a smaller model having the resources to process/interpret a larger model (or subset thereof).

[ Parent | on HN ]