"In another experiment, they asked Astra to make Qwen3.5-0.8B better at math by editing its weights"

??

Which it did by doing an inefficient version of the aforementioned 1940s algorithm.

Essentially modern LLMs are able to do normal AI engineering tasks, like running optimization algorithms over weights. The media has been inaccurately describing this as if the AI had some special insight into the weights, when it reality it just performed the tasks an human engineer might do.

Right, humans don't need some special insight into weights either, since it's just a bunch of numbers that you can analyze like any other table with numbers. However, if you dump them on someone like in this prompt: https://github.com/RikiyaT/lm_understand-logs/blob/main/exp1... they might complain about the formatting or ask whether they can use Excel instead of doing calculations by hand. (The experiment with Qwen3.5-0.8B didn't use the "dump weights into the context as numbers" approach.)

That's not what's being described here; that's described in another paper that this is a followup to.

I think the point being missed is that this same tool that can do the 1940s optimization algorithm can also translate Ancient Greek to English, develop a web app and draw a picture of a pelican riding a bicycle. That’s really crazy.

No, the article we are discussing here used a GPU solver implementing the primal-dual hybrid gradient method, which cannot do any of those things. (This is not the same thing as the simplex algorithm from the 01940s. I was wrong about that.) The LLM discussed in the previous work was not in fact able to do that itself. Like a normal person, it wrote a Python program to do it.

While the apparent advent of AGI is indeed really crazy, the point of this article is sort of the opposite.