Re:The problem with metrics is a big problem for AI
by minimaxir on Saturday October 19, 2019 @03:00PM
(#21300092)
Metrics will always be gamed as long as a) there is an incentive to game them and b) there are no proportionate consequences for gaming said metrics.
To use YouTube video recommendations as an example, (a) is reflected by the current title/thumbnail brinksmanship seen recently as well as the shift to "alt-right" topics, and (b) is reflected through YouTube contorting itself trying to justify not punishing such methods. In the case of recommendation rabbit holes (e.g. alt-right rabbit holes), it's why journalism highlighting the trends is especially important as a correcting force.
To use YouTube video recommendations as an example, (a) is reflected by the current title/thumbnail brinksmanship seen recently as well as the shift to "alt-right" topics, and (b) is reflected through YouTube contorting itself trying to justify not punishing such methods. In the case of recommendation rabbit holes (e.g. alt-right rabbit holes), it's why journalism highlighting the trends is especially important as a correcting force.