Re:Fine-tuned small LLMs can beat large ones with programmat...
by simianwords on Tuesday August 05, 2025 @12:37AM
(#44794393)
I think its a good idea but how do you not accidentally benchmark hack here?
[ Parent
| on HN ]
Re:Fine-tuned small LLMs can beat large ones with programmat...
by GabrielBianconi on Tuesday August 05, 2025 @10:48AM
(#44798715)
We set up dataset splits and the usual best practices. Of course, if you overdo things, you can still hack benchmarks; our goal isn't to publish SOTA numbers but rather to illustrate results from our methodology. We didn't even tune hyperparameters, we just used the default choices. Definitely a valid concern for teams chasing SOTA though.
Thanks!