News for hackers. Stuff that matters.
Re:Bad numbers in the “gzip beats BERT” paper? by godelski on Monday July 17, 2023 @03:31PM (#36762858)
What's also good to see here is that the post '90 numbers usually don't even fall within the error bars of the pre '90 numbers. While uncertainty is great, it isn't the end all. I think a lot of people forget how difficult evaluation actually is. Usually we just look at one or two metrics and judge based on that, but such an evaluation is incredibly naive. Metrics and measures are only guides, they do not provide certainty nor targets.

There are no comments on this comment yet.