Samsung found itself at the center of a scandal yesterday, after an AnandTech investigation suggested the company had rigged the firmware of the Exynos 5 Octa version of the Galaxy S4 to obtain better scores in a series of benchmark apps.
Specifically, it was discovered that, when running AnTuTu, Quadrant and other benchmarks, the phone’s GPU was unlocked to run at 533MHz, instead of the maximum 480MHz frequency that apps can normally achieve. The CPU was also said to be set for top performance when benchmarks were being run. As a consequence, benchmarks scores of the Samsung Galaxy S4 were artificially inflated.
As evidence for the allegations, AnandTech supplied both empirical observations and fragments of code found in the phone’s firmware, including a function called “Benchmark Booster.”
Samsung has just issued a response to the allegations, which were picked up by many top technology websites and caused quite a bit of uproar in the Android community.
The official response is a post on the company’s Korean Samsung Tomorrow blog. According to the folks at The Verge, who seemingly had access to a human translated version of the Korean text, the gist of the post is: “[We] did not use a specific tool on purpose to achieve higher benchmark scores.”
Samsung defends itself by claiming that all apps that run in full screen mode, or with no status bar visible, get access to the top speed of the GPU, which is 533MHz. These apps include the browser, camera, video player, and benchmarking tools. In other words, Samsung says that many apps, not only benchmarks, run at 533MHz, and that’s the normal behavior.
“We did not use a specific tool on purpose to achieve higher benchmark scores”
Update: here’s Samsung official response in English:
Under ordinary conditions, the GALAXY S4 has been designed to allow a maximum GPU frequency of 533MHz. However, the maximum GPU frequency is lowered to 480MHz for certain gaming apps that may cause an overload, when they are used for a prolonged period of time in full-screen mode. Meanwhile, a maximum GPU frequency of 533MHz is applicable for running apps that are usually used in full-screen mode, such as the S Browser, Gallery, Camera, Video Player, and certain benchmarking apps, which also demand substantial performance. The maximum GPU frequencies for the GALAXY S4 have been varied to provide optimal user experience for our customers, and were not intended to improve certain benchmark results. Samsung Electronics remains committed to providing our customers with the best possible user experience.
However, there are a few things that Samsung fails to explain. For one, both Brian Klug of AnandTech and AndreiF, the modder who alerted the site about the discrepancy in the first place, stated that no other app beside the whitelisted benchmarks reach the top speed of 533MHz.
Another rather damning piece of evidence that Samsung fails to address is the presence of the “Benchmark Booster” functionality in the Galaxy S4’s firmware and the whitelist of benchmarks that were hardcoded to gain access to top speeds.
So, who’s lying? I tend to think that AnandTech, one of the most reputed technology sites, has the truth of it. Samsung’s terse and vague comment fails to provide a credible explanation for the situation, and at least in my book, the Korean company is still on the hook.
To address a few comments on yesterday’s post, the practice of “optimizing” firmware to achieve the best possible scores in benchmarks is not new, and is likely that other mobile manufacturers have been using similar tricks. And it’s true that the real life consequences of these tricks are very limited.
But that doesn’t mean we should close our eyes when underhanded things like this happen, especially when it comes to Samsung, the leader and public face of the Android world.