After Samsung has been proven to boost the performance of the Galaxy Note 3 in benchmark applications, the South Korean company went on record to say it doesn’t cheat, but not before Apple’s Phil Schiller noted the “shenanigans” on Twitter.
A couple of days ago, Ars Technica discovered that the Galaxy Note 3 behaves differently when a benchmark app is detected in order to maximize scores, and was able to prove that the boost only happens when certain well-known benchmarking apps are recognized.
Soon after that article was published, Apple’s Phil Schiller took to Twitter to post a link to it, writing a single word in the tweet – “shenanigans” – a word also used by the article’s author:
— Philip Schiller (@pschiller) October 1, 2013
— Philip Schiller (@pschiller) October 1, 2013
On the same day, Samsung provided a short comment to CNET UK to clarify the matter, saying it doesn’t cheat on benchmarks:
“The Galaxy Note 3 maximises its CPU/GPU frequencies when running features that demand substantial performance,” Samsung said in a statement to CNET UK.
“This was not an attempt to exaggerate particular benchmarking results. We remain committed to providing our customers with the best possible user experience.”
Interestingly, AnandTech posted an article in which it showed that almost all device makers with the exception of Apple and Motorola (Google) have such boosting methods in place for better benchmarking results.
At the end of the day, this is just business as usual in the mobile world. Of course Samsung denied the accusation; you wouldn’t expect it to acknowledge that it has set up the Galaxy Note 3 to score better in benchmarks – for what it’s worth the device was proven by the same Ars Technica to surpass its competitors when the performance boost is removed.
And of course Apple reacted to the news, especially since it came at this particular time, when the new iPhones and Galaxy devices are battling in stores for the buyer’s attention.
With all that in mind, we have a new question for you today: what are your thoughts on benchmarks?
Like this post? Share it!
How can they say it was not an attempt to exaggerate benchmark results when its written into the code to exaggerate benchmark results?
They are lying
Assuming there are benchmark apps for iOS, are any of them as thorough as those for Android? I don’t see how they could be, with all of the restrictions Apple places on third party apps.
3dmark, Gfx benchmark and geekbench are all their
all companies cheat, lie, and steal/copy what’s new?
so you mean Samsung and others should spared for cheating and fooling customers.
apple, yes certainly. “we do not copy everyone copies us”. and yet they do copy freely from everyone around. the apple shill calling out on this one is ridiculous, just means that they do care about benchmarking – even though they do not benchmark – getting into this muddle , just proved they do care about benchmarking, going around the town with dung to splatter on opponents is a good example why.
they just showed Samsung true form , just that’s it.
notifications. NUFF SAID
is there patent for notification. until then what about ios inteface , siri clone ,etc.
what IOS interface ? grid of icons ?
siri clone ? so are you going to take US patent office patent extortion racket as a measure of innovation ? lol
You forgot to mention “bounce effect” and “global search”.
that’s what I meant.
in doing that, they also showed they are so full of themselves
or they where right for suing samsung.
which they pretty much won only in the US, and lost elsewhere, revealing uncle sam helping them out.
As ruled by the patent troll who was jury foreman in the trial of Samsung’s alleged stealing of “dimensional shading on rounded corners on icons” (standard to any art program – Macromedia would sue everyone – if they were bought out by Adobe!) and yet, the USTC admits the only rerason Apple has a embedded tablet is because they stole it from samsung. I think these lawsuits are stoopid.
they didnt sue for rounded corners , google it for what they sued samsung using which patent.
design patents that covers iPhone’s features such as the “home button, rounded corners and tapered edges” (US D593087) and “On-Screen Icons” (US D604305).
grid of icons – how ridiculous.
rounded corners – patenting geometry as innovation, how ludicrous
there is just lot to that you haven’t read case study .
rounded corner / grid of icons were priot art – that was my point. you asked me to prove about rounded corners – I proved it.
apple still are scumbags.
square enix used rounded icons and gradients before apple. I just dont see apple ownership on a art style or design
my sega game gear had rounded corners.
“there is just lot to that you haven’t read case study ”
oh please, plenty of places where apple’s stupid patents have been smashed to to smithereens. Look at EFF/ ACLU/ Groklaw or such places for some neutrality on the issue – everyone else are just apple shills
No I won’t because in my word documents I made things with rounded corners and gradients – back in 1997! I saw these on my orientation slides in my first powerpoint at Williams Sonoma Incorporated, in 1998. We all thought rounded corners and gradient shading (available for free) were as hot as that lens flare in Photoshop. I don’t understand how a device “aesthetic” can sue, unless you are Euclid or Archimedes. The design of most phones are pretty basic and simple. i can’t wait for a guy who made the toaster to go hey, “your phone design is black and rounded rectangle – that looks like the bottom of my toaster! The screen is even where the crumb tray indention is!!!” The other thing I LOVE about this pic – is the “home icon” on the Samsung. First of all, comparing the App Drawer to the home screen of an iPhone is not comparing Main interface to Main interface. Most people do not navigate their android phone from the App Drawer, nor is it their main view. The App Drawer is rank and file of all the apps on the device, alphabetically or customizable. The Home Screen is customizable. The Home Screen on an iPhone is as far as i can tell – organized by one time Steve Jobs threw random objects at a dartboard and saw which ones stuck. This is not nearly as serious as the confirmed ruling that Apple stole Data Embedded tablet technology from Samsung – and would never have had a tablet with Data plan without stealing it from Samsung. That is theft of technology without licensing – much worse that something NATIVE to draw objects in Microsoft Word, and designed by Macromedia. Macromedia OWNS this look because they made the software for it. Everyone started putting gradients on everything.
its not about stealing , but replicating a lot to the device , thats why apple was angry and sued.
“we do not copy everyone copies us”
Where did you get that quote from? Your mouth?
yes, dimwit. I was opining.
On the one hand you have Samsung allegedly tweaking code to boost benchmark performance, and on the other hand you have Apple prohibiting customers from something so trivial as choosing their own default browser or email client. Six of one, half a dozen of the other…
cheater and control freak are different animals
Note 3 is the best phablet right now in the market…
I can’t believe sammy went so cheap again…ofc Sammy has done similar cheap tricks before…I don’t think they will stop !!!
“I can’t believe sammy went so cheap again”
and you are ALWAYS surprised. standard samsung tactics. even apple stoops lowly with the help of the gubment. 9 out of ten times they lost lowballing android worldwide, in the stateside, uncle scam helps them clubbing opponents with them patents
if it would have been apple caught cheating this would have been opposite , everyone would just go on rage , now because this is android so its ok.
Anyone who knows better wouldn’t give a shit.
you are saying that now.
Sad but true. Its even sadder when I heard almost every major manufacturer did it…*EXCEPT* Google/Motorola & Apple. I really expected better from you guys, HTC & ASUS!!
Apple was not tested, as far as I know.
you are becoming s-sheep now dude have you even read anandtech article.
Yes, I have read. The word Apple happens only once in a sentence you cannot conclude if they tested iOS. Apple is not in the list, what suggests it was not tested, since Motorola is there with all columns set to “N”.
How about read the author’s comment too?
Anand Lal Shimpi :”Apple doesn’t do any of the frequency gaming stuff, no.” October 2 comment from AnandTech.
so, why are they bothered to come out to crap on samsung ? ‘cos they belong in the same cesspool!
The author did mention other manufacturers. Samsung won the headline because it’s a repeat offender (with the S line) and biggest cheater. Looking at the chart, only Note 3 managed to tweak just about every benchmark out there. Sad really, Note 3 is a top notch product, one of the best out there, and the product will sell itself regardless of the benchmark scandal. The fact that they still try to cheat shows ethical and corporate culture problem.
my point being , why does apple bother with this petty fights between android makers ? they hold themselves in high esteem for not being in the pissing contest. if they really did not care they wouldn’t have got into this in the first place.
A coment is not a technical report.
apple is on the list look at the chart, man you are just now lying.
I am getting tired. Apple iPhone 5s appears on benchmark charts, but explain to me WHY Apple devices are not listed in the chart showing which manufacturer and model has or has not cheats?
Motorola is in that chart with an “N” in all columns. Others have an “S” here and there. But Apple isn’t tthere. I can only conclude they could not affirm if Apple was cheating or not although they “suggested” it is not. As I said the word Apple appears only once in that article.
so you want to say anandtedch is lying , hmmm.
Why did they make an entire article then? They could just affirm every one is cheating but Motorola and Apple. And if someone doubted, their answer could be “so you think we are lying?” and the objective answer to that would be “where’s the proof?”. My point is not if Apple is cheating or not. I think they’re not, yet, because there is not much to do with a dual core as opposed to quad and octa cores. My opinion is that their affirmation (in a comment, not in the article itself) that Apple doesn’t cheat has no support in this article. You have the right to blindly believe Anandtech’s words but I don’t.
I read your other comments and concluded that you are an i-sheep.
I am mot defending Samsung since all major manufacturers are involved.
Apple just can’t be caught cheating because their source code is closed and copyrighted. You cannot mess with it without being sued.
no , there kernal is open source google it.
The kernel is not the operating system.
What , you dont have a clue.
darwin is the kernel IOS is based on. after collaborating on it a bit of time, one of the reasons why open darwin ceased (july 2006) is because in the end apple made it their hub to develop OSX related products (mooching off ) , apple made it impossible for open darwin to take off, they were there only to create something for themselves with the expertise of the community. however, once their goal was met, they just took off. they do dump the source, but its irrelevant now.
as regards you pointing out to JJG – I think you need to explain why you think the kernel is the OS. for example linux is only the kernel, GNU-linux is the entire package. similarly, you can’t do much with only the kernel, you still need the system resources – graphics subsystems/ audio/ windowing system
http://www.opensource.apple.com/source/xnu/xnu-2050.24.15/ here is your answer man ,research before you claim , even if they are lying have you ever seen there iphone or ipad lagged like samsung smartphones they have lived upto the performance and smoothness or they have ever touted about specs, you people just bash apple for anything even its not there fault.
first please explain how the kernel is the OS.
next, i know what you have just posted. they’ve been working on multiple project including the webkit and other stuff.
i gave you the example of how they played around with darwin, and none of it is a lie.
apple needs to be thrashed where they deserved to be.
I should not, but I will help you with your research… Click on “parent directory” several times until you reach this main page: http://www.opensource.apple.com/ you will see that what you saw was OS X related, not iOS related.
I have an iPhone 3G that lags as hell since it was updated to iOS 4. That’s just what’s happening with the iPhone 3GS and iPhone 4 with iOS 7.
If you read the article Samsung didn’t cheat on anything. They actually optimize the phone so that when intense programs run on it, the cores adjust to that applications needs. What’s wrong with optimizing the phone to do that? Doesn’t that make sense? It does the same thing when you play an intense game on it. The cores adjust to what the game needs. This is just a bunch of crap.
but games also dont run at that frequency except for benchmark only.
Yes, the apps may run like that in at least two sceneries:
1 – Custom ROM;
2 – Stock ROM rooted with a CPU booster app, with which you can select active cores and min and max clock frequency.
setcpu must be new to him
but we are talking about stock kernal and samsung cheating , where setcpu comes here.
I was replying to “Johnny Jay Gotit”. Once you have rooted your device, you get full control and you can either throttle or push the device all the way to maximum permissible range. He was countering you saying the device could indeed be pushed to the max, just like the supposed “artificial” boost. heck, people have always been pushing their devices – CPU/ GPU/ even RAM.
you are really a resident apple troll here, and you are accusing him to be a shill when all he did was offer real world use cases (although it might be just the enthusiasts)
Apple defenders choose convenient sceneries. If we were talking about Android flexibility and customization options you would come here with your mouth of jailbreak.
just let it go , either you are paid by samsung or trolling.
The day when u OC tour CPU for gaming you will ser the diferente! ;)
You never will do it with an iphone!
that day your battery will die in an hour and phone will overheat , i know that. done a lot on s2 and gaming on iPhone is much better due to optimization and support.
ActuallyiI own a GS2 skyrocket CM10.2 OC and is holding off pretty good the beating!;)
me too but got bored.
You obviously never read Anandtech’s analysis.
Go read the article again, you really didn’t pay good attention.
And Apple fans would be called sheeple for not caring about the technical details of benchmarks.
We’ve all learnt not to take anything crapple says with a grain of salt
when you say “not” to take anything they say with a grain of salt. it’s a double negative. meaning, we shouldn’t take what apple says as “not true”. it’s like you’re are saying apple always says everything truthfully.
next time you try to troll, troll with correct grammar you peasant.
Maybe that’s precisely what he means.
That we need to pay more attention to what Apple says.
Wait, at first I thought Samsung was falsifying their score of the benchmark. But the device actually hits the benchmarks by its own hardware? I don’t think there’s any cheating in that :/
The cheating is that said boost never applies to other tasks. So the benchmark doesn’t show real world performance (not unlike any other benchmark there is
Hmm what if it does ._. What if its like Intel Turbo Boost. Maybe during gaming or high-performance computing, the performance does get boosted. Maybe not for long periods but still.
Idk if they found out they are cheating it should be because it doesn’t boost anywhere else
Same app + different package name = different results = cheating.
Your wrong sir any 3d graphic intensive game would have the GPU and CPU maxed out on any device to ensure a stutter free performance. I don’t think they are cheating at all they just encrypt the software to recognize 3d graphic intensive applications. Nothing cheating about that
Idk dude I don’t even care I don’t look at benchmarks anyways. I was just explaining what I understood from the reading
You understood wrong. Don’t comment and stop caring when you find out you’re wrong.
I mean if they say it’s cheating it can’t be the way Samsung says, don’t you think?
Plus, the way they found out was they took a popular benchmark (Geekbench), changed the name (to 6eekbench, from “G” to “6″) and since the system didn’t recognize the app scores lowered significantly, if it’s the exact same test results should be the same (around the margin of error), again, don’t you think?
If you want me to keep commenting I can, I just didn’t want to start a war about something I don’t really care about when I was initially just explaining, not bashing
The problem is that the test results aren’t the same? I don’t get what your trying to say in this comment. The problem is exactly what you just mentioned. The same app, different package name, different results. That’s cheating. Especially when other apps like games don’t get the same level of treatment. Even highly intensive apps or games aren’t given the same priviliges (i.e. cheats) as these benchmarks.
That’s just what I said, what did you think I meant?
Uh, just noticed we meant the same thing yeah. It was I that understood wrong, sorry man. Think I confused your comment with the one of @jjordan.
No, he isn’t wrong since they’ve cheated on many benchmarks apps, including 3d graphic intensive ones.
In order to show that they cheat, they used the exact same app, yet just changed their “package name” (which is like an identification of it).
If you were right, the results should be about the same between the 2 versions of the app, since they are identical in all aspects, and the device should have used its best performance for the task given to it.
I agree. The trick was waking up the idle cores and maxing the clocks from the begining of the test, not in the middle.
I’m currently using the note 3 and all it seems to do is crash like no tomorrow :/
Coming from an iPhone 5, i really did expect better. Now I understand why apple is so popular.
You’re doing it wrong
How so? I cant seem to go 5 min without it pausing, continued with a ‘unfortunately *specific app* has stopped’
Dafuq did I just read?
The article did not except Apple. They excepted Motorola after testing. But they did not test Apple, as far as I know.
No, both Motorola and Apple didn’t cheat (and also nexus devices) .
“With the exception of Apple and Motorola, literally every single OEM we’ve worked with ships (or has shipped) at least one device that runs this silly CPU optimization.”
False. Apple is tested
Read the entire article and tell me what kind of tests were made with which Apple product. You will not find this answer, since the word Apple happens only once in the entire article and it is in the sentence quoted above.
Benchmark shmenchmark, if you’re smart enough to understand what benchmarks are, then your smart enough to know that they often don’t matter in real-world use. Regardless of the benchmarks on the Galaxy S4 and LG G2, they both run much better with Nova Launcher on them over their own launchers. The Nexus 4 also is extremely smooth even though there are phones with much better specs. Complete fact.
After Samsung lying about the Note 3 being unlocked and sim-free when in fact it is not, I’m not going to trust them. Let them drown in their own ****.
I guess speedtest.net is a shenanigan too since my ISP boosts QOS for speed tests. iLag 7 is a shenanigan, just install it on any iPhone 4/4S or iPad then you will see the very definition of lag.
Does Phillip even have any sense in his head,no wonder he is from apple? LOL…feel so rush to call that one to Samsung, that doesn’t even apply to other tasks,plus it comes from anada tech and apple insider, they just said apple did not do that,really? where’s the apple device’s result? pretty fishy in this article.
Anandtech has personally vouched that Apple does not cheat in benchmark tests. His word is good enough for me.
Semms fishy to me buddy,they are all pure apple fans and trust me. It is all make-up one to bash apple’s competitor,but guess what, apple will still be a loser ,no matter what!
That means you only had to read the title. It says everything you need to know.
Benchmarks are supposed to get you a notion of how good the hardware of the device is , how well it performs on any app.
When the device acts with a boosted performance only on certain number of apps, which include benchmark apps, that’s cheating for me.
I think it’s wrong. I also think it’s weird that Samsung has done it, since they have a good device already. Only reason for them is to be on the top of the list for a little longer than usual.
I hope there will be a “Hall of Shame” for all devices found to be cheating.
I also wonder if installing custom ROMs removes the cheating factor, whether they are based on Samsung’s stock ROMs or AOSP. My guess is that it will occur only on those that are based on Samsung’s stock ROMs .
Another pointless article from Arstechnica about benchmarks… and of course “San Apple” is clear as a whistle… Of course they get the sympathy of Phil…
As i said on the Google+ post about this, i said this……….
Ehm, wouldn’t it be a good idea to allow the heavy running apps to actually use the available power of the device that the phone comes with, aka a 533 MHz GPU and a 2.3 GHz CPU?
This is the same as using a Koenigsegg Agera R witch have a 1115 hp engine. That engine can only produce those 1115 hp if it runs on E85 bio fuel. If it runs on normal 95 octane fuel, it will only produce a little over 900 hp.
Source on that is here: http://www.supercars.net/cars/5261.html
Does that means that everyone who want to run the car on the race track (benchmark) and use the E85 bio fuel (Samsung’s script) is cheating over the others who are using normal 95 octane fuel (no Samsung script) on the street?
No, it wouldn’t. The car are still good for 1115 hp accodring to the specs in the same way as the Galaxy Note 3 are good for 2.3 GHz on the CPU or 533 MHz on the GPU in it’s specs. It’s just that some apps can’t use the E85 bio fuel (all available power) over others who can use that. So they will run on the 95 octane fuel (clocked down a little) instead.
Just because some apps can’t do that, it doesn’t mean the Galaxy Note 3 is cheating or overperforming, it’s just that there are some apps who can’t use the full power of the phone because the apps isn’t powerfull enough to use that.
I hope that this explains why all the fuss about this is nothing more than whining and crying.
I think the issue here is that what you are proposing is fundamentally a very bad idea.
Even if they could, I doubt most people would want to run their phone’s CPUs at max capacity due to heat concerns, and the damage it could do to their phones’ internals in the long run. Nor should such an option even be available, as it would do more harm than good.
Most mobile software isn’t even optimised for quad-core processing to begin with. I personally feel the “right” solution is to go back to the drawing board and start optimising dual-core processors (Apple has shown that when properly done, a dual-core processor running at a lower clock speed actually outperforms a less optimised quad-core processor with a higher clock speed) and users also benefit from longer battery life.
Unfortunately, I think that with the general mindset that more cores = better performance (and with Samsung upping the ante with its octa-core processor), this is an uphill battle.
Best comment on this subject.
People can’t get over the fact that benchmarks in and of themselves are “artificial performance tests.”
How is it different to optimize hardware for a specific set of apps vs “benchmark apps” vs any other software running on the device?
If they can max the hardware for a “test” who cares? All that shows is that the hardware can potentially do X. . .
And that’s why reviewers should run identical “real world tests” that compare each devices “real world” performance for “real world end users.”
“It’s not cheating if everyone else does it. It’s rampant cheating.”
As you apple only say shit bout android because there loosing the market an on comparison apple still only use dual core as were top android devices us quad core and apple still use Samsung for its chip maker
Never mind that Apple’s dual-core processor is smoking pretty much every other quad-core processor in terms of performance and battery efficiency?
hmm no it does not, even according to benchmark (different platforms iOS has the advantage for being less resources demanding) and yet it doesn’t beat lets say an S800, but clock per clock and core to core yes it does, which expected since the A7 is a generation ahead, (the next flagship will leapfrog the A7 as they’ll have ARMv8 instruction sets + Cortex A57 architecture and (64bit support which matter less) as for power consumption I don’t know it it will be more powerefficient or not that is to be seen
rarely do all cores ever run on a quad core. . . so, we really have no real world comparisons.
nor does it really matter as anyone who wants to use an idevice needs to be ready to accepts apple’s lifestyle — clearly only a small percentage of the population is willing to accept that.
the reality is, any current SoC is plenty good for smartphones at this point in time — other than that it’s a pissing contest with little real world benefit to users.
iDont care, i love ANDROID with Pride!
I don’t know why this is even being discussed but here my points of view on the matter.
-If you check Anandtech view on the matter, not all of the OEMs cheat or rather they don’t do it the same way : where as HTC and Sony for example reduce idle time in two benchmarks : AnTuTu and Velamo where as Samsung does for ALL famous benchmark suites.
-The other thing is, is that not only Samsung stops it SoC from idleing in all Benchmarks or almost, it overclock it Exynos SoC beyond their advertized capcity and their TDP which is not only dangerous for the device,it is plain cheating.
-Also it was proven in the past that Samsung coded it software to cheat benchmarks such as Quadrant
If it boosts the performance while running benchmarks, which test how much it can do, then it will do the same with any game that needs all of it’s power. That means that it’s not “cheating at benchmarks”, it’s just reserving some power until it’s needed. Most likely a battery saving “feature”
Sigh, benchmark boosting has been around for several decades now. Remember the good ol’ days of graphics cards benchmark boosting? Remember when everyone used Quake2 as the standard, and all it took was a rename of the .EXE file and suddenly boom, different scores?
This isn’t new, and exactly why synthetic benchmarks should always be taken with a grain of salt. Everyone is always going to do some greasing under the hood to make their product look the best. The *true* benchmark is your own human interaction with the device.
Unfortunately tho, it takes quite some time to really test out a device. Most buyers don’t spend that much time on testing.
Anyhow, why should Schiller care? Should he not be heads-down, looking at his own products and improving them? Why even call any kind of attention to the competition, whether good or bad?
to show that Apple doesn’t cheat (or so it seems)
WHAT! Samsung cheats? Nah, I think it’s called optimizing. If they do cheat thoug, it’s still insignificant to me. I still buy a Samsung…only if it priced like a Nexus, haha.
a benchmark tester is a drag strip, not the regular highways (and parking lots!) we normally drive on.
If the Note 3 is faster than the oh so amazing 64bit IPhone 5s Apple must be p*ssed off lol
samsung always cheats
no brain, what’s your name? no brain no name
@Schiller…jealousy makes you nasty…
I don’t see how Samsung “cheats”. Sure, they maximize the performance of their hardware during benchmarks…but that is like saying that a person is cheating in a race when they devote 100% of their energy to their performance.
People just like to hate on Samsung for being the best.
I think the word cheating is a little harsh, if Samsung ran ten tests on the same phone they would most likely get ten different results. Now what Samsung chooses to do with those results is up to them. So they only post their best results I don’t think that is exactly considered cheating. Unless an independent agency did all testing on all the different devices and posted them all then that would be the only way it could be deemed totally fair. Its very impressionable survey to start with. So again, cheating is a very strong word for being leanient. Choose your words wisely is all I can say.
Samsung product a cheap on sales
Check this benchmark without cheat https://play.google.com/store/apps/details?id=com.fetxpinc.cpumark