The point of Fine-tuning small models is for specific downstream tasks, which SOTA models can do, but at higher costs. It is purely an economic play, not an attempt at pushing boundaries of SOTA.
If the appropriate qualifiers were added to say "cheap model X does better at test Y than expensive model Z when we fine tune X to take Y test of existing knowledge" then it would be a more accurate statement, but naturally less impressive.
People who are not really into it, don’t care.
What if you have to do the task a billion times? Which model will you choose?
It doesn't mean frontier models aren't good at harder tasks.
A fine-tuned small model is like a laser pointer. It only illuminates a tiny specific spot. But it can illuminate it as brightly as the huge floodlight, for a tiny fraction of cost.
If we were to take these at face value, why is it that the frontier labs' models are making legitimate new discoveries (e.g. Erdős and Jacobian conjectures) and these models are not?
To me, a better signal of capability would be similarly performing novel work at the same or better level, which they presently are not. I say this as someone who very much looks forward to open models being more capable, but to deny the gap is misguided hopeful hype.