Astra can read and write weights to improve Qwen

Re:Astra can read and write weights to improve Qwen by bilsbie on Monday November 02, 2026 @03:03PM (#50342853)

GOOD Point by my wife. Why would LLM’s have any understanding of weights? It wouldn’t be in their training data.

[ Parent | on HN ]

Re:Astra can read and write weights to improve Qwen by woah on Monday November 02, 2026 @03:31PM (#50343274)

Why not?

[ Parent | on HN ]

Re:Astra can read and write weights to improve Qwen by slopnt on Monday November 02, 2026 @05:25PM (#50344839)

it probably is. its very likely they train their models with weights for some tasks

[ Parent | on HN ]

Re:Astra can read and write weights to improve Qwen by tiborsaas on Monday November 02, 2026 @08:02PM (#50346613)

LLM-s operate in a higher dimensional space and they produces a 1D stream of tokens. We think of the output as all there is to how they work.

It seems like that raw weights are still within the natural language of an LLM and they can easily map weights to a latent space representation, do a transformation and map it back.

I feel that asking about training data is missing the point, they might all be able to do it naturally like I can move in 3D compared to a 2D being living on Flatland.

[ Parent | on HN ]

Re:Astra can read and write weights to improve Qwen by ex-aws-dude on Monday November 02, 2026 @08:22PM (#50346777)

It is in the training data

People discuss weights on the internet

You can also have a model generate output and then just check what was activated

[ Parent | on HN ]