No, it describes a linear optimization algorithm from the 01940s editing a Qwen model with billions of parameters.

"In another experiment, they asked Astra to make Qwen3.5-0.8B better at math by editing its weights"

??

Which it did by doing an inefficient version of the aforementioned 1940s algorithm.

Essentially modern LLMs are able to do normal AI engineering tasks, like running optimization algorithms over weights. The media has been inaccurately describing this as if the AI had some special insight into the weights, when it reality it just performed the tasks an human engineer might do.

Right, humans don't need some special insight into weights either, since it's just a bunch of numbers that you can analyze like any other table with numbers. However, if you dump them on someone like in this prompt: https://github.com/RikiyaT/lm_understand-logs/blob/main/exp1... they might complain about the formatting or ask whether they can use Excel instead of doing calculations by hand. (The experiment with Qwen3.5-0.8B didn't use the "dump weights into the context as numbers" approach.)

That's not what's being described here; that's described in another paper that this is a followup to.

I think the point being missed is that this same tool that can do the 1940s optimization algorithm can also translate Ancient Greek to English, develop a web app and draw a picture of a pelican riding a bicycle. That’s really crazy.

No, the article we are discussing here used a GPU solver implementing the primal-dual hybrid gradient method, which cannot do any of those things. (This is not the same thing as the simplex algorithm from the 01940s. I was wrong about that.) The LLM discussed in the previous work was not in fact able to do that itself. Like a normal person, it wrote a Python program to do it.

While the apparent advent of AGI is indeed really crazy, the point of this article is sort of the opposite.

I don’t understand the negative reactions to your post, it’s a good insight.

Basically the weight edit approach is find a semantic direction for the undesirable behavior and steer away from it. With access to the weights I see no reason a model (even one as large as Astra) couldn’t apply that principle to itself. This probably includes at runtime if you wanted to give the model write access to itself

Agreed. You could look at a previous session which got the right answer eventually and make edits until the path to the right answer was more direct.

It's not clear if this would generalize to other related uses, or how disastrous it would be to unrelated uses, but it's worth studying. Feels like a sort of agentic Lamarckism.