This certainly isn’t the first time OnePlus has been caught cheating in benchmarks. This time, the company’s latest flagship, OnePlus 5, is under fire.
Why and how do OEMs cheat on benchmarking? – Gary explains
Just a few months ago, XDA-Developers discovered that the OnePlus 3 and OnePlus 3T had been programmed to cheat in benchmark apps in order to maximize their scores. Essentially, the CPU in these devices remained at 1.29 GHz for the big cores and 0.98 GHz for the small ones, and this minimum frequency reduced the number of step frequencies, resulting in lower variance. The result was slightly better performance in benchmarks, but the publication concluded that it was due to the sudden merge of the OxygenOS and Hydrogen OS development teams. OnePlus issued a statement then, promising that these mechanisms will not be present in future OxygenOS builds.
OnePlus is manipulating benchmarks with its latest smartphone, but this time, it’s more 'blatant' and 'consciously introduced.'
Fast-forward to June, OnePlus has unveiled its latest flagship, the OnePlus 5. While benchmarks should never be taken as an absolute indicator of a device’s performance, the OnePlus 5 did manage to create buzz when leaked benchmark results showed it outperforming popular phones like the Galaxy S8 Plus and Pixel XL. Well, it turns out those numbers may not be telling the whole truth. XDA-Developers reports that OnePlus is again manipulating benchmarks with its latest smartphone, but this time, it’s more “blatant” and “consciously introduced.”
XDA-Developers received its review unit ten days ago and has learned that it contains a blatant cheating mechanism that maximizes performance within benchmark apps. It states that other review units are likely to be affected and warns potential buyers that benchmark results reported by other online publications are probably based on misleading data.
We’ve reached out to OnePlus for comment and will update this post when we hear back.
To be more specific, the team has found that the OnePlus 5 “resorts to the kind of obvious, calculated cheating mechanisms” to “maximize scores.” Because all little cores are affected and kept at 1.9 GHz, the OnePlus 5 is able to achieve some of the highest GeekBench 4 scores of a Snapdragon 835 device to date.
Because all little cores are affected and kept at 1.9 GHz, the OnePlus 5 is able to achieve some of the highest GeekBench 4 scores of a Snapdragon 835 device to date.
They state that AnTuTu, Androbench, GFXBench, Quadrant, Nenamark 2, and Vellamo are also affected – the exact same list of apps as last time when the OnePlus 3 and OnePlus 3T were involved. Without the cheating mechanism, the team found that only 24.4 percent of readings returned the maximum frequency of 1.9 GHz in the small cores whereas with it enabled, the number jumped to a whopping 95 percent.
As you can see above, the difference is only noticeable in multi-core results, which is what most people look at, given Android’s full support of multi-threaded applications. The manipulation of the smaller cores inside Snapdragon 835 results in enough of a boost to put the OnePlus 5 ahead of its competition. XDA-Developers has received an official comment from OnePlus since its publication:
People use benchmark apps in order to ascertain the performance of their device, and we want users to see the true performance of the OnePlus 5. Therefore, we have allowed benchmark apps to run in a state similar to daily usage, including the running of resource intensive apps and games. Additionally, when launching apps the OnePlus 5 runs at a similar state in order to increase the speed in which apps open. We are not overclocking the device, rather we are displaying the performance potential of the OnePlus 5.
In contrast to the official statement issued during the OnePlus 3/3T scandal, in which the company admitted its wrongdoing and promised a fix, this time around, OnePlus defends this mechanism and argues that it is simply to show the potential of the device. However, as XDA-Developers points out, in no way is the overclocking of the little cores an accurate representation of everyday usage.
Of course, OnePlus is not the only company to have been accused of benchmark cheating: major Android OEMs have been in the same boat. However, the difference is that OnePlus is a company that survives and thrives on its fan base and community. With deplorable software support, increasing prices, and now even more benchmark controversy, OnePlus may be headed in the wrong direction. What was once thought to be a disruptive force could end as a failed attempt if the company loses its identity.
What are your thoughts on OnePlus and its latest benchmark-cheating scandal? Has OnePlus lost its magic? Has it become too similar to companies like Samsung? Let us know in the comments below.