The State of Cheating in Android Benchmarks

http://www.anandtech.com/show/7384/state-of-cheating-in-android-benchmarks

by sciwiz • 13 years ago

43 5 13 years ago

This is rather important from a moral standpoint! I'm surprised no one is talking about this!

siglesias• 13 years ago

Some have made the objection that phone benchmarks should test the CPU and removing the underclocking is entirely appropriate.

Isn't this somewhat intellectually dishonest though? A phone is a system, and how fast the CPU can run given the constraints of the system is a testament to the manufacturer's hardware and software engineering abilities: the software's power management, the hardware and ID's thermals. OEMs shouldn't give themselves a free pass on this. We know how fast a snapdragon can go. Now let's see how fast you can make it go in your handset.

zobzu• 13 years ago

Note that its not just about coming out on top of benchs.

its also making you believe you gotta change phone because the new ones are so much faster.

I wouldn't be surprised if next time we see 20% improvement "due to cheating" instead of "just 10%".

dak1• 13 years ago

Or just switch what tests you're using. You could keep alternating between 2 tests and have 20% performance improvements year over year on the exact same hardware.

philthesong• 13 years ago

Android is a failure. I know this comment might be offending for someone.

The State of Cheating in Android Benchmarks

Related Stories