Bad numbers in the “gzip beats BERT” paper?

Re:Bad numbers in the “gzip beats BERT” paper? by 1024core on Monday July 17, 2023 @02:03PM (#36761740)

True. You want to always use an odd number so there are no ties.

I'm guessing they were trying a parameter sweep, and found that (thanks to the bug) they got the best results for K=2.

This too is problematic in its own sense.

[ Parent | on HN ]

Re:Bad numbers in the “gzip beats BERT” paper? by ks2048 on Monday July 17, 2023 @04:37PM (#36763708)

Yes, agreed. One small point: for the multi-class case (more than just two classes), which include all the datasets here, you can still get ties for odd k. e.g. k=3, you can get 1 vote each for 3 different classes, etc.

[ Parent | on HN ]

Re:Bad numbers in the “gzip beats BERT” paper? by 1024core on Monday July 17, 2023 @05:11PM (#36764172)

Multi-class is trickier. Maybe we can break down an N-class problem into N binary-classification problems?

[ Parent | on HN ]