News for hackers. Stuff that matters.
Re:30% of Google's Emotions Dataset Is Mislabeled by codedokode on Thursday July 14, 2022 @05:10AM (#32093731)
You can filter out bad workers by preparing an additional well-labeled dataset and removing those who made even a single mistake in it. Also you can give the same task to several workers and check if they give the same label. However this won't protect against a bot using multiple accounts and giving answers based on a hash of a question so that the same question gets the same answer in every account.
Re:30% of Google's Emotions Dataset Is Mislabeled by stjohnswarts on Thursday July 14, 2022 @10:55AM (#32096822)
"removing those who made even a single mistake" means you don't want humans working on this lol. Humans will always make mistakes.