> companies employ grad-student descent (hire a bunch of PhDs and task them with improving MLPerf results)

No joke, always cheaper than massive optimization tasks. I’ve managed a team working on automl for years and I can say with certainty, the cheapest option is always hiring interns (grad or bachelor students) or a hackathon