News for hackers. Stuff that matters.

A comment on 30% of Google's Emotions Dataset Is Mislabeled. Parent comment

Re:30% of Google's Emotions Dataset Is Mislabeled by amelius on Thursday July 14, 2022 @10:15AM (#32096357)
> I wonder how others do this kind of thing. Assuming I have 100k text blurbs I want to classify of sentiment and/or emotion and want at least 95% accuracy, what are the options?

Use the consensus of 3 or more annotators (or median).

[ Parent | on HN ]
Re:30% of Google's Emotions Dataset Is Mislabeled by wongarsu on Thursday July 14, 2022 @10:40AM (#32096644)
But is paying 3x $5/h getting better results than hiring a local collage student for $15/h?
[ Parent | on HN ]
Re:30% of Google's Emotions Dataset Is Mislabeled by sgc on Thursday July 14, 2022 @11:14AM (#32097063)
Won't you just wind up with a collage from either source?

Answer is yes, because those $5/h workers are likely just as educated, but from a less affluent part of the world.

[ Parent | on HN ]