Samsung denies benchmark gaming allegations in vague response [Updated]

July 31, 2013
After Samsung was accused yesterday of gaming benchmarks in order to obtain better scores for the Galaxy S4, the Korean company issued a vague response that fails to provide a credible explanation.

Massive improvement in AnTuTu benchmark scores

Samsung found itself at the center of a scandal yesterday, after an AnandTech investigation suggested the company had rigged the firmware of the Exynos 5 Octa version of the Galaxy S4 to obtain better scores in a series of benchmark apps.

Specifically, it was discovered that, when running AnTuTu, Quadrant and other benchmarks, the phone’s GPU was unlocked to run at 533MHz, instead of the maximum 480MHz frequency that apps can normally achieve. The CPU was also said to be set for top performance when benchmarks were being run. As a consequence, benchmarks scores of the Samsung Galaxy S4 were artificially inflated.

As evidence for the allegations, AnandTech supplied both empirical observations and fragments of code found in the phone’s firmware, including a function called “Benchmark Booster.”

Samsung has just issued a response to the allegations, which were picked up by many top technology websites and caused quite a bit of uproar in the Android community.

The official response is a post on the company’s Korean Samsung Tomorrow blog. According to the folks at The Verge, who seemingly had access to a human translated version of the Korean text, the gist of the post is: “[We] did not use a specific tool on purpose to achieve higher benchmark scores.”

“We did not use a specific tool on purpose to achieve higher benchmark scores”

Samsung defends itself by claiming that all apps that run in full screen mode, or with no status bar visible, get access to the top speed of the GPU, which is 533MHz. These apps include the browser, camera, video player, and benchmarking tools. In other words, Samsung says that many apps, not only benchmarks, run at 533MHz, and that’s the normal behavior.

 

Update: here’s Samsung official response in English:

Under ordinary conditions, the GALAXY S4 has been designed to allow a maximum GPU frequency of 533MHz. However, the maximum GPU frequency is lowered to 480MHz for certain gaming apps that may cause an overload, when they are used for a prolonged period of time in full-screen mode. Meanwhile, a maximum GPU frequency of 533MHz is applicable for running apps that are usually used in full-screen mode, such as the S Browser, Gallery, Camera, Video Player, and certain benchmarking apps, which also demand substantial performance. The maximum GPU frequencies for the GALAXY S4 have been varied to provide optimal user experience for our customers, and were not intended to improve certain benchmark results. Samsung Electronics remains committed to providing our customers with the best possible user experience.

However, there are a few things that Samsung fails to explain. For one, both Brian Klug of AnandTech and AndreiF, the modder who alerted the site about the discrepancy in the first place, stated that no other app beside the whitelisted benchmarks reach the top speed of 533MHz.

Another rather damning piece of evidence that Samsung fails to address is the presence of the “Benchmark Booster” functionality in the Galaxy S4’s firmware and the whitelist of benchmarks that were hardcoded to gain access to top speeds.

So, who’s lying? I tend to think that AnandTech, one of the most reputed technology sites, has the truth of it. Samsung’s terse and vague comment fails to provide a credible explanation for the situation, and at least in my book, the Korean company is still on the hook.

To address a few comments on yesterday’s post, the practice of “optimizing” firmware to achieve the best possible scores in benchmarks is not new, and is likely that other mobile manufacturers have been using similar tricks. And it’s true that the real life consequences of these tricks are very limited.

But that doesn’t mean we should close our eyes when underhanded things like this happen, especially when it comes to Samsung, the leader and public face of the Android world.

Comments

  • Daniel Fleming

    Wow Samsung, code doesn’t lie and andantech shows the code in a hex editor of what apps are allowed to operate at the higher speeds and it’s only benchmarking one’s. Somehow I don’t think that code just appeared by accident.

    • rjr162

      I’d take andantech’s input very carefully… Instead of re-typing it all I’ll repost my concern with some of their comments:

      “AnandTech spotted a similar behavior when it comes to the CPU – when running GLBenchmark 2.5.1, AnTuTu, Linpack, and Quadrant, the cluster of Cortex A15 cores @ 1.2GHz kicks in. When running GFXBench 2.7 (a benchmark that wasn’t whitelisted in the firmware), the lower-powered A7 cluster clocked at 500MHz is used.”

      You know, just throwing this out there.. but the first benchmarks listed under using the A15 cores actually, you know, benchmark the SYSTEM.

      GFXBench is just as it says.. a GPU benchmark written to test OpenGL and such, which should be hardware off-loaded via the GPU.. so not taxing the CPU, which means WHY would the A15 cores kick in if the A7 cores could handle the load just fine?

    • http://forum.xda-developers.com/member.php?u=2926289 Jasonwsc

      Someone please enlighten me, why would you want your GPU running at full frequency for browser, camera and video player? Are those apps really that demanding?

      • SkylaC90

        No you don’t want to be enlightened trust me, come and join the resistance! brother.

        • http://AndroidAuthority.com/ Bogdan Petrovan

          Ingress nerd alert!

          • SkylaC90

            Right!!!! that’s my game, the lazy and anti social need not apply

      • RarestName

        So that other applications from the Play Store cannot perform better than the stock ones?

        • http://AndroidAuthority.com/ Bogdan Petrovan

          This. Unless they are benchmarks.

      • Piyush

        Have you played any highend games on 1080p devices , because it doesn’t run on full frequency for highend games.

  • Jack Parker

    It really doesnt effect real world performance at all, and if the GPU can be clocked to that speed, which is pretty damn high, good for them. it just means that we could over clock the GPU as and when we need, not just for benchmarks

    • Piyush

      wut

  • Blowntoaster

    I think Brian and AndreiF should check the rest of the code/test and see if the GPU does overclock for fullscreen applications.
    Maybe the Benchmark Booster code was added for a reason as without it the GPU would not max out in testing. Did you check that Anandtech/AndreiF?
    Give Samsung the benefit of the doubt before jumping on the horn…

    • http://AndroidAuthority.com/ Bogdan Petrovan

      I can’t speak for anybody, but Brian just retweeted this: “Benchmark apps are full screen’ is a pretty hilarious excuse.”

      • jdoee100

        I don’t know what is your beef with samsung, but your source is one person. I would wait until Brian’s accusations can be confirmed by another. And, samsung’s explanation is not ambiguous, it’s a credible explanation. Since when did Brian’s tweet became an end all?? Very biased article and comments.

        • sfasljkas

          yeah aa has been pretty critical of samsung the last few months i noticed it as well… and one article said maybe its better if samsung fell down a notch or two so that other android oems can “catch up” like wtf lol

          • Mike Palmer

            I don’t get all the Samsung hate and bias attitude……..after all its Android vs Apple, I don’t know why people have forgot that Samsung is Android. I guess if one person has aa opinionin the tech world the rest have to follow!

          • Sean Karpa

            Samsung may be Android, however, Samsung does try to distance itself slightly from Android, to ensure their phones continue to stand out.

          • Mike Palmer

            Yeah it makes sense to me. seems like these sites used to have articles against Apple end how closed ios is and how over prised they are and now it’s usually only ever about Samsung…

          • number29

            Samsung are now too successful = time to bash them to all hell and back. People really really really don’t like successful companies I’ve established.

          • Mike Palmer

            How dare a company try to make products to make money……… They should give them away for free lol

          • Apple_Nexus

            What’s Android vs Apple? You do live in a fantasy world don’t you.

            Newsflash: Samsung lying. What next, earth is round and the sky is blue?

            If they’d said something along the lines of we just wanted to test the theoretical max speed of the device and coders must’ve left that code it’d be more believable but Samsung’s answer just digs a bigger hole for their BS.

          • mobilemann

            It’s not apple vs anything idiot. And you can’t have a “bias attitude” although you can be biased.

            comparing this site to anandtech or ars, is a joke

          • Mike Palmer

            So your not a idiot for name calling over a tech site? lol

          • mobilemann

            It’s “you’re” idiot.

          • number29

            If anything, it’s Apple vs Samsung vs HTC vs Nokia etc etc.

          • feedback12

            Interested in your focus on defending Samsung. It has taken over Android and it is now coming to light that it has been using shady practices to get to the top including copying, fake reviewers, fake benchmarks, fake negative reviews for HTC and more. There needs to be an exposé on this company.

          • Mike Palmer

            Really? Don’t think I have ever seen a review that was bad about HTC lately. Anything from the past that was said was probably true because they left their consumers behind and people don’t forget!

        • http://AndroidAuthority.com/ Bogdan Petrovan

          Funny how just three weeks ago I was accused of being pro Samsung.
          I am not saying that Brian Klug knows all, I just respect his opinion and the arguments that he brought.
          About Samsung’s explanation, I don’t find it credible. It’s not that I have something with them, but they fail to explain why there’s code named “Benchmark Booster” inside the firmware and why there are benchmark apps that are *hardcoded* to reach better speeds.
          Thanks for reading.

          • jdoee100

            “Benchmark booster” is there because samsung wants to put out the best foot forward like anybody else would.
            This is from the verge, “but the site(Anandtech) did note that other benchmark apps that are not explicitly mentioned in code were also behaving the same way.” This supports samsung’s explanation, that other programs are also geared for higher speed, not just the “hard coded(or hacked, as you like to call it)” benchmarks.

          • http://AndroidAuthority.com/ Bogdan Petrovan

            Honestly, I don’t want to debate this anymore. You’re entitled to your own opinion.

          • Sean Karpa

            Referring to your “putting your best foot forward.”
            What if dell released a PC and said the CPU will run at 5ghz but it comes clocked at 2.5ghz. That’s the difference. Any phone can run faster, but we want to know what it will actually run at during day to day usage.
            Depending on the truth of this, Samsung has no shame.

          • jdoee100

            Compare apples to apples, Samsung never said that their phone runs at 533MHz, but it clocked at 480MHz. Besides PC industries have “optimized” their chips for a long time, it’s nothing new for PCs. Moreover, recently intel has done this with their chip, claiming their chip is a lot faster than Snapdragons, but in reality they “hacked” their program to run around the benchmark test.
            Even if samsung’s explanation is false(which I don’t think it is), all big corporations(intel, Nvidia, and countless others) are in the same category. Why samsung gets all the blames and shame while others get free passes is puzzling. Bias, jealousy, fear?? who knows?

          • Sean Karpa

            The other companies don’t get a pass, at least from me. I bash just about every company, certainly all the technology companies. I love bashing them, except HTC, they are dying at the moment, so I won’t attack them while they are in the hospital.
            By making the phone give performance that is not true in the real world, Samsung is the saying that this phone will run like this always.
            You make it sound like I said it was a bad decision for Samsung to underclock it. You drew a conclusion here that has no basis; I never said that. I understand the purpose for underclocking and it commonly used by every company. There is nothing wrong with that; however, for the company to mislead people on the performance of the phone? Yes, that is my problem. It is a dishonest thing to do (not that every other company isn’t dishonest at some point). The whole argument of “every other company does it” is a silly one with no good supporting. So Samsung is just every other company? We should settle for that?

          • jdoee100

            I never said what samsung supposedly did was okay. If indeed samsung did hack the program to just increase benchmark scores, then that is a wrong. However, I have problem with singling out samsung when other companies have done the same or worse. For example, people like to call samsung phones plastic, or cheap build. Most phones are made of plastic, but samsung is the only phone being called out for plastic. Samsung has great build quality, but people love to call out samsung’s cheap build quality. This case is the same, plenty of others have done it, but it’s the samsung who’s called out.

          • Sean Karpa

            One thing you will see is that the top makers always get beat up. When Apple totally ruled as far as high-end smartphones (up until about the introduction of the S3 IMO) they were always getting negative attention from haters. I see now that Samsung is on the same level as Apple (in a different way, in the sense of more flexibility and certain features), Samsung is getting that negativity.
            Like when Apple didn’t shock the world with 99969969% growth, they were murdered in the stock market and “lost their way.” Same for Samsung recently, they didn’t post a 54606945% growth, thus……
            People chose sides and fight back. Simply as that.
            BTW. Samsung’s phones do feel cheap when against HTC One, iPhones, and most Motos. So since Samsung is suppose to be the best (at least in their way), they fall short here. Especially since Samsung’s commercials paint the S4 to be “the next big thing.” Which I think is a stuck-up thing to say.
            But anyway, that negative attention will be around for a long time.

  • Blowntoaster

    I think Brian and AndreiF should check the rest of the code/test and see if the GPU does overclock for fullscreen applications.
    Maybe the Benchmark Booster code was added for a reason as without it the GPU would not max out in testing. Did you check that Anandtech/AndreiF?
    Give Samsung the benefit of the doubt before jumping on the horn…

  • 2gerry

    What a somewhat annoying article. Ehether its Samsung or any other manufacturer. They all have the possibility of doing this. Not that they actually do. Its business. Put that in your mind. Its all about Sales. Put that in your mind too. After everything is said and done, whether true or not, at the end of the day, all that matters is…for us consumers, is the product good enough, is it fast enough to do what you want it to do, is it worth your cash, are you happy with the product.

  • Lil bit

    That excuse…. Considering the existing evidence and then blame it on full screen benchmarking, is it possible? Samsung just went full retard, they made things worse with this statement. Reminds me of Bill Clinton, sleeping with that chick was not bad, it was his lies that took him down the hardest.

    • SkylaC90

      still one of the greatest presidents in these United States. furthermore he went on to finish his term as President.

      • http://AndroidAuthority.com/ Bogdan Petrovan

        Bill Clinton rocks!

        • SkylaC90

          I know right, he’s such a good leader and so inspirational.

      • Mike Palmer

        Didn’t take him down that much huh lol

        • Apple_Nexus

          I believe it was Monica that went down… I know, this joke sucks!

          • Eki

            I believe it was Monica that did the sucking. Your joke was awesome.

  • Roberto Tomás

    You can’t tell when responses are not very specific but it sounds like function they found was only one of many functions, and so its whitelist excluded other software that does run at the full speed. It should actually be pretty simple to test it, if a few specific examples of software that would get whitelisted besides benchmarks were provided by Samsung.

  • lil bit

    If Sammy was smart they’d say it was a slip, an internal test function used in-house before launch that was forgotten and should not have been in the release firmware.

  • Aliyu Abba

    It works with other apps too at 533MHz, the 480Mhz is there to protect your phone against crashing, heating etc for prolonged usage, all the drama for nothing

    • Piyush

      THAN WHY FAKE BENCHMARK. please stop the nonsense.

      • Aliyu Abba

        What do you mean fake, it was clearly stated the GPU runs at 533MHz, just because they found a whitelist doesn’t mean other apps don’t use that speed, video player, browser, gallery, camera all use the 533MHz, so stop hating.

        • Mike Palmer

          Any reason to hate Samsung and everyone jumps on the band Wagon……. how dare they make products to make money lol

          • Piyush

            same as every other product , pour that truth

          • Mike Palmer

            For sure they are all companies just trying to make the most money they can and compete with each other.

          • Apple_Nexus

            It’s not really “competing” when Samsung are doped up on race day, is it?!

        • Piyush

          who told i am hating , because of this other product didnt get the much praise needed.

      • wikwakcow

        yeah, I agree with you. as for my understanding, antutu, quadrant, etc are not user experience apps. they are a benchmarking apps. then, why did samsung put them on the whitelist?

  • Balraj

    Still sammy fails to explain the benchmark booster presence…….

  • OMGgary

    !!

    • Bryan Z

      lol

  • Dan

    I am not surprised. Remember when Intel faked a demo of their HD Graphics? It turned out that the “game” being “played” was just a VLC video and the Intel guy was just faking it.

    Or when browsers were being optimized for passing artificial tests and benchmarks but have no real-world use?

    Or when AMD and NVIDIA were also optimizing benchmark performance over actual gaming performance?

    Samsung is just being a dick and following industry standard for optimizing (even cheating) for benchmarks just so it’ll look more impressive than it is. I own two Samsung smartphones and a Tab 2 7.0, and I am looking forward to getting a Note 8.0. This revelation may not change my mind, but it pisses me off when the company needs to cheat to bolster its test scores.

  • John Mortimer

    Hi, it clear why they do this, its so the better core are used for the speed tests or the crap cores may be used, that’s all its there for only to make the tests uses the right cores only, so they say auto Max out the CPU so it uses the big main cores

  • http://www.flowerstation.co.uk/ Flower Delivery Guy

    give Samsung a break. The have over the last few years absolutely destroyed Apple. My neutral opinion thinks Apple are right now way behind in innovation and design. gordon @ flowerstation.co.uk

    • number29

      Apple have been destroyed?! Just now I’ve been able to visit their website and pick from a whole range of products!

      • Jack Parker

        You mean the iPhone, iPad and Mac PC’s and Laptops? Wow, so many products

        • number29

          Right, so they’re not destroyed.

          • Jack Parker

            I think what flower meant was, not destroying their products but their legacy and innovation, at first you bought an iPhone because it offered more what the average smart phone had at the time, but years on and apple have had the same UI, the same design and quiet frankly their selling the same device year after year

          • number29

            I think you’re confusing innovation with iteration. Samsung and Apple both iterate. Samsung iterates a lot more than Apple. Apple does a lot more innovation than Samsung (the last innovative product launched in the mobile devices spectrum was the iPad, and prior to that the iPhone). Samsung does very little innovation in the mobile space; they simply follow a formula that was developed by someone else.

          • Mike Palmer

            I’d have to say I don’t agree with you this time, look at the Note series they created a while new kind of phone the phablet “terrible name tho”. And allot of new products that Samsung are making are all new.

  • rijoenpial

    Ok, have read the news and some ops down below, so here’s my adding to the confusion: Why would a korean company place on the S4′s firmware in plain english ‘Benchmark Booster’, knowing full well about rooting and firmware watchdogs and such? My answer is that this name is misleading, and surely Samsung’s explanation is credible enough… Also, news are facts, not interpretational biased takes by one or two bloggers, who are not journalists at all, just opinion givers… The tone of the news is misleading and the explanation given by Samsung and many others in this forum show that sensationalist one-sided news mixed with personal takes into what those words actually mean, are not news, but op-eds… Having said this, much as in PC benchmarks, it is the performance of the game that matters, not nitpicking frequencies for geek talks… To me, if a game works and works flawlessly, who gives a crap about benchmarks? I know, those who value x over y because x has 100 more points in GEEKbench (very illustrative name) and whatever! Gimme a break! If both perform flawlessly, nitpicking becomes tech geeks fave passtime! And I am not offending anyone, as I am one myself! And proud of it! Cheers

  • rijoenpial

    And the reason why they’re vague about it? Because they couldn’t care less about geeky things like ‘oh, this game actually only runs at 35 frames per second and not the 36 they said!’ or ‘This benchmark clocked at four hundred whatever instead of the five hundred alleged!’ God, I am a geek, but sometimes, this nitpicking tires me! The important thing is how REAL it performs, in the REAL world, not in artificial statistical environments!

    • CoolDude

      So why LIE?

  • Martin Cataldi-Rogers

    you know what who cares

  • Chris Boots

    Shouldn’t think most users could give a monkeys fart to be honest

  • seyss

    fail. these asians dog eaters are a bunch of profiteers. cant trust em

  • Leonardo Rojas

    Oh no, I was going to get the i9500 for my on-contract upgrade here in Peru as silly Lenovo is delaying too much with its K900. But now I feel devastated, with no hopes for incredible smooth high graphics gaming anymore! And I don’t know if I still get it (cause it’s the best option here) if I will be able to get over this idea and forget that its GPU is not running to its maximum speed *^* I’d go paranoid and see lags here and there and won’t be able to enjoy my games at all @^@!! What have you done Samsung!!!
    LoL

  • http://www.mobileosworld.com/ Xen Lee