Samsung stops skewing benchmarks results with KitKat update
Recently Samsung and other Android manufacturers started to artificially boost their numbers in benchmark apps, but it appears that practice is starting to come to an end with Samsung’s KitKat update.
In its version of Android 4.3 Jelly Bean on the Galaxy S4 and Galaxy Note 3 Samsung included code that forced all cores in the phones’ processors to run at maximum speed, skewing benchmarks higher than normal. After updating the phones to Android 4.4.2 KitKat, however, Ars Technica found the processors now run at normal speeds when a benchmarks app is launched.
The publication was able to confirm its findings with John Poole of Primate Labs, the makers of GeekBench. Primate Labs put some detection code into GeekBench that triggered whenever phone maxed out its processors artificially when the app was running. While the Samsung phones running Android 4.3 Jelly Bean triggered the code, after the upgrade to Android 4.4 KitKat it did not.
The change may seem small, but it is somewhat significant. Many smartphone reviews include benchmark tests as an easy way to compare the phone being reviewed to similar phones. Speed can be the one differentiating spec when you’re comparing two phones that are alike in almost every other way. Artificially setting processors to max during benchmark tests skews the results and makes it so they don’t represent real-world usage.
Even after removing the code that skewed the results the Galaxy S4 and Galaxy Note 3 saw better benchmark tests under KitKat than when they ran Android 4.2 Jelly Bean. While the update does bring some new features, it also makes Android more efficient and faster, which is always nice.
Samsung isn’t the only company getting rid of the artificial benchmark boosting. There are reports that Sony is getting rid of similar code in its KitKat updates for Xperia phones.
Are you glad Samsung is no longer skewing benchmark results? Do benchmark results even matter to you?