Previously the only way to improve a neural network (short of giving it more data) is to change the size (width, depth, and layers), add stuff like dropout, change the loss function, etc. That amounts to a few dozens of levers to control millions or billions of weights. You're trying to navigate blindly through a latent space with thousands of dimensions and millions of weights to find some optimum point with only a handful of navigation tools. If you can instead have a system reason with all the weights and dimensions directly, you now have millions of levers at your disposal.

It's not that it's more impressive than solving Millenium Prize problems. It's that if a model can edit its own weights, then it can self-improve without having to undergo training, which means it can self-improve extraordinarily faster.

This is very dangerous because it allows AI capabilities to grow at a much faster rate than humans are able to keep up with.

It will just give the people better understanding. And the extent of the improvement is yet to be seen, it's likely that once you get all the obvious low hanging fruit it would be very hard to improve, the possibility space is just too wide.

What if every model under a ~1B size gets super cracked over the next 2 months? What could swarms of tiny agents 2 ooms larger than what seen so far do?

If thats keeping you upright in your chair then buckle up, wait to the distributed systems version of the tech, its coming.