News for hackers. Stuff that matters.
Re:Reverse-engineering the problematic tail behavior of Five... by pedrosorio on Sunday October 25, 2020 @01:07PM (#24887829)
If they were wrong, but “less wrong” than all others, you should pick their model (unless you have an oracle, because the alternative - flipping a coin or “relying on your intuition” is rarely better).

Also, it doesnt make sense to look at a single prediction to evaluate a model.

Out of all the predictions they have made (did you look at individual state predictions?), how many were correct (and how confident were they?) - how many were wrong (and how close to 50% were they?).

That is how you evaluate a model (aka cross entropy)

There are no comments on this comment yet.