Re:The State of Cheating in Android Benchmarks
by siglesias on Wednesday October 02, 2013 @05:24PM
(#6485505)
Some have made the objection that phone benchmarks should test the CPU and removing the underclocking is entirely appropriate.
Isn't this somewhat intellectually dishonest though? A phone is a system, and how fast the CPU can run given the constraints of the system is a testament to the manufacturer's hardware and software engineering abilities: the software's power management, the hardware and ID's thermals. OEMs shouldn't give themselves a free pass on this. We know how fast a snapdragon can go. Now let's see how fast you can make it go in your handset.
Re:The State of Cheating in Android Benchmarks
by dak1 on Wednesday October 02, 2013 @10:36PM
(#6486766)
Or just switch what tests you're using. You could keep alternating between 2 tests and have 20% performance improvements year over year on the exact same hardware.
Isn't this somewhat intellectually dishonest though? A phone is a system, and how fast the CPU can run given the constraints of the system is a testament to the manufacturer's hardware and software engineering abilities: the software's power management, the hardware and ID's thermals. OEMs shouldn't give themselves a free pass on this. We know how fast a snapdragon can go. Now let's see how fast you can make it go in your handset.
its also making you believe you gotta change phone because the new ones are so much faster.
I wouldn't be surprised if next time we see 20% improvement "due to cheating" instead of "just 10%".