News for hackers. Stuff that matters.
Re:HellaSwag: 36% of this popular large language model bench... by TillE on Monday December 05, 2022 @10:22PM (#33875487)
I'm not entirely clear on what ActivityNet is (one of the primary sources for HellaSwag), but it looks like amateurish descriptions of videos, like you would write for audio descriptions for the blind, except written very badly.

I'm guessing it's just Mechanical Turk content which wasn't even spellchecked.

Re:HellaSwag: 36% of this popular large language model bench... by PraetorianGourd on Monday December 05, 2022 @10:27PM (#33875527)
Agreed, but I am concerned over the chance that lots of these models are used in critical scenarios without much validation of the data set.