Yes, they can reward hack. However, making sure the model knows what your requirements are (e.g., "do not quantize, make sure to evict, meet this set of specifications") can mitigate that, and simply looking at the code briefly (or having the LLM describe it back to you) can mitigate it quite easily. Often, such issues are a result of insufficient specificity by the user. If you tell it just to make the model faster, it'll do it and not assume additional requirements.

Same thing applies to benchmarking, a few guardrails goes a long way. As long as you aren't completely vague ("set up benchmarks", vs. "write tests over a variety of common shapes, be careful to control for environmental variables") the models do quite well. I've checked over the code it's written and it's all fairly reasonable. I do mention cache eviction in the article, this generally wasn't an issue and the model did this automatically on the first pass (back in Opus 4.6 days). I think you might not be giving them enough credit.

I'm not completely familiar with the writers of this article, but I've never had an LLM try to hack this, even sometimes telling me a target was impossible without quantization or approximation:

"Code based analysis already tells a detailed story, but there is an overlooked parameter: "boundaries of the hardware". " - the coreauto post

This project has spanned several models since Opus 4.5. I think even then such details were trivially known by them, thought they required much more handholding than they do now. I'm honestly not sure how the article writers ran into the hacks they did.