Re:Pretraining on the Test Set Is All You Need
by SubiculumCode on Saturday October 28, 2023 @11:21AM
(#38050487)
There were certain language choices in the paper that led me to think that just maybe they weren't doing what they were doing, but the longer I read, the more certain I became that they were doing exactly what they shouldn't, wasting my time.
[ Parent
| on HN ]
Re:Pretraining on the Test Set Is All You Need
by maleldil on Saturday October 28, 2023 @11:38AM
(#38050645)
It's satire. They're criticising "good" results obtained by trained on contaminated datasets.
Re:Pretraining on the Test Set Is All You Need
by SubiculumCode on Saturday October 28, 2023 @12:00PM
(#38050825)
well yes, so said the discussion. I was just irritated that i read theough it that far.
but not being a machine learning pro, I'm happy that I knew enough to be highly suspicious.
Re:Pretraining on the Test Set Is All You Need
by marcosdumay on Saturday October 28, 2023 @12:18PM
(#38050982)
It's a 3 pages paper with absurd ideas from the title on.
Imagining it was not clear enough, the author decided to name the model something 'pronounced “fictional”', and shows how it can answer questions that weren't even asked yet with perfect accuracy right on the abstract.
Accept that as an exercise of familiarizing yourself with the area's language. If you are interested on it, it's not lost time.
Re:Pretraining on the Test Set Is All You Need
by naniwaduni on Saturday October 28, 2023 @03:30PM
(#38052673)
It's a two-and-a-half-page paper with about a page and a half of total text with an obvious joke title, absurd and obviously joking claims in the abstract, and names the model "fictional". This is all before you even click through to the PDF!
If it takes you reading the whole paper up to the point where the author explains the joke to realize that the paper tagged satire is satire, I mean, I'm not sure paper-reading is for you. There are unmarked papers out there that aren't even satirical, just wrong!