You just can't go changing the layers of a neural network independently of each other if you are doing guided optimization.

For a start, it's not even a given if increasing some weight will increase or decrease the result. The neurons are all tightly coupled.

You can do it on unguided optimization, what is one of the reasons I strongly suspect our brains use something similar to simulated annealing.

Sure, we don't know the exact details (Geoff Hinton spent much of his career trying to answer this question), but at the big picture level it does seem clear that the cortex is a prediction engine that minimizes prediction errors by feedback, and most likely does so in a localized way. Exactly how these prediction updates work is unknown.

Could you expand a bit on how you think simulated annealing could work?

Hum... I have no model for what actually happens. It's just that simulated annealing appears quite often on the biologic world, we never found any coordination communication channel that could create something more "intelligent" (AFAIK, there are just a few "neighborhood to global" analog signals available), and there isn't a multiplicity of attempts interacting with each other (again, there is no channel for that) that would lead to a parallel evolutionary strategy.

Also, when we learn a mechanical task, our errors tend to cool down around our successes, instead of following a direction.

But again, I have no idea of any mechanism that is there. It's more because all the others seem to have been eliminated.