ML is like a genie in that it is very good at optimizing exactly what you tell it to, and so you need to think long and hard about what you're telling it to optimize, because it might turn out to be bad proxy for what you really wanted.