Well, there's also nonconvex optimization, which is not exactly easy. On the other hand, "just use Adam" mostly works...