Re:Astra can read and write weights to improve Qwen
by bob1029 on Tuesday November 03, 2026 @03:02AM
(#50349311)
If we are being slightly reductive, this is just more of the exact same thing. The token stream in this case happens to be a projection of the weights of a neural network. I don't see where we violated any of the prevailing assumptions about how these systems operate.
The mapping between spaces in this case is mostly linear. This isn't like gzip or a Turing machine. I'd argue this is even more permissive (easier) than the LLM's originally intended use case of human language.
What happens if we can come up with a model that can directly emit a frontier LLM given some seed context? I feel like we could train a model to recreate astra verbatim. At some level it's just bytes on a disk that can be bundled up into tokens. As long as the representation is rational and consistent I don't see why this wouldn't work.
The mapping between spaces in this case is mostly linear. This isn't like gzip or a Turing machine. I'd argue this is even more permissive (easier) than the LLM's originally intended use case of human language.
What happens if we can come up with a model that can directly emit a frontier LLM given some seed context? I feel like we could train a model to recreate astra verbatim. At some level it's just bytes on a disk that can be bundled up into tokens. As long as the representation is rational and consistent I don't see why this wouldn't work.