Re:Astra can read and write weights to improve Qwen
by bushido on Monday November 02, 2026 @04:53PM
(#50344366)
This is exciting. I've been training a Qwen3.5-2B model for the past few weeks, trying to teach it certain skills that it lacks.
In the course of testing, one of the things which has been consistent is that when setting a thinking token budget for relatively small prompts that require a limited response, a pretty healthy token budget needs to be set because the model spends the first few hundred tokens just repeating the question/prompt back with some boilerplate "Analyze & Report" type language. I was also running into some loop problems sporadically.
After reading this article I asked Astra to edit the backbone so that it started actual reasoning faster and applied a similar looping fix. It was able to reduce the token waste before reasoning by about 20%. I haven't been able to get it to loop yet, so hopefully it fixed that too.
My message is admittedly low on info. But it's a pretty promising start!
In the course of testing, one of the things which has been consistent is that when setting a thinking token budget for relatively small prompts that require a limited response, a pretty healthy token budget needs to be set because the model spends the first few hundred tokens just repeating the question/prompt back with some boilerplate "Analyze & Report" type language. I was also running into some loop problems sporadically.
After reading this article I asked Astra to edit the backbone so that it started actual reasoning faster and applied a similar looping fix. It was able to reduce the token waste before reasoning by about 20%. I haven't been able to get it to loop yet, so hopefully it fixed that too.
My message is admittedly low on info. But it's a pretty promising start!