Except Apple.
Issues like the Intel i9 performance going significantly downward with consistent usage are often missed by benchmarking apps: https://arstechnica.com/gadgets/2018/07/we-tested-throttling...
However given that Apple isn't the only one who licenses their designs I'd put a large sum of money on them doing the same thing.
My 3 year old Oneplus X is more responsive than my iPhone 8 (work phone).
GPU drivers are still loaded with such replacements, either to work around bugs or to optimize games where a developer partnership was impossible or otherwise rejected. I think that with the shift to lower-level GPU APIs this practice is beginning to fade, but it's basically been par for the course since programmable shaders became a thing.
However, I agree that shader replacement is great for the consumer especially in situations where developers have agreed to vendor lock-in/specialization agreements. It's only questionable to me in the sense of benchmarking applications.
https://www.hardocp.com/article/2001/10/23/optimizing_or_che...
TLDR: ATI did the same thing with Quake 3 Arena which was commonly used as a benchmark game for testing new video cards of that era and which many game news sites were using.
It did go too far, even to the extreme of Nvidia hardcoding in their GPU driver the exact camera path that 3dmark took ( https://www.extremetech.com/computing/54154-driver-irregular... ), which is more like what the mobile space is grappling with now. That was solved through a combination of scolding and by just changing the benchmarks to be what people actually do. You don't see 3dmark show up in GPU reviews much, you instead see performance results in a handful of major games. If mobile follows suit and shifts to doing things like benchmarking the average/90th/99th percentile of a game of Fortnite or something then these cheating practices become less feasible as a whole.
Yep. That was the problem at he time. There were no shaders/etc so you’d just taking a demanding game and see how good it ran. No one optimized specific games in their drivers.
So what ATI did made it appear the GPU was stronger on average that it was.
If it was disclosed it would have been a feature and might have been welcomed. But it was secret.
I remember seeing discussions about whether the kind of optimizations we see today are cheating. They sort of are. But why wouldn’t I want the faster frame rates?
I forgot about the nVidia “micro optimization for a specific benchmark” one. No good excuse there because no one actually plays it.
Well, not really. It really did run Quake 3 at the speed the benchmarks were reporting. The problem was it got that speed at the expense of a drop in image quality, and a somewhat substantial one at that.
But if you launched the game and just played it you did get the exact same performance that the benchmarks were reporting.
> I remember seeing discussions about whether the kind of optimizations we see today are cheating. They sort of are. But why wouldn’t I want the faster frame rates?
Desktop GPU optimizations or mobile ones? Desktop GPU optimizations these days are not considered cheating because of 2 key points: 1) it improves the actual game experience (this is why reviews focus on game performance now) and 2) it doesn't impact image quality (periodically you'll see image quality comparisons to ensure both sides are still being honest)
The mobile things that Huawei are doing though are cheating because it does not translate into improvements in actual user experience or gameplay. It's tuning the thermal & power consumption to hit the throttling point at the exact duration of the benchmark. This doesn't translate in any way to any real-world improvements. Thus, cheating.
They'll happily ignore thermals, run custom shader microcode and any other tricks that'll boost their benchmark scores on common apps.
We always kept a set of internal benchmarks that we didn't share with vendors and used that as a part of SoC evaluation. It had the benefit of not being gamed and also lined up with our normal workloads so we had a decent idea of where we expected the whole system to land.