I mean, maybe this is where neural networks that forget are valuable? They allow for exploration, and don't create local maximas in the same way?