Re:Google T5 scores 88.9 on SuperGLUE Benchmark, approaching...
by lidHanteyk on Thursday October 24, 2019 @10:07PM
(#21351172)
Yes. This is more generally known as Goodhart's Law[0]: when a metric is used as a goal, then people will game the metric in order to win, making the metric useless.
There is no fundamental way to overcome this problem, except by not using metrics as goals.
There is no fundamental way to overcome this problem, except by not using metrics as goals.
[0] https://en.wikipedia.org/wiki/Goodhart's_law