BDTi upcoming ‘user experience rating’ aims to measure real-world device performance

August 16, 2013
16 4 6

galaxy s4 vs galaxy s4 active aa benchmark

Whenever the latest smartphone scores a killer benchmark rating, it absolutely guarantees that the handset is the biggest and baddest around. It also means there is noticeable smooth-as-butter performance during day to day use. Okay – maybe not.

In reality, benchmarks often don’t reflect real world use. Instead, benchmark software measures theoretical performance, without factoring in things like software optimization. Even worse, scores can easily be manipulated, making the generated number even less meaningful.

Luckily, a solution could soon be at hand, at least if BDTi (Berkeley Design Technology, Inc) has anything to say about it. The California-based company has formally announced plans to create a consumer-focused rating system that plays by a different set of rules than your typical benchmark software.

No matter what, the most important factor for determining the right device will always be a combination of reviews, hype and just how well a particular device fits into your needs.

According to BDTi’s president, Jeff Bier, this means the new user experience rating will focus on aspects such as battery life, transfer speeds, application performance and other results that really matter to the average consumer.

Of course announcing plans to release such a rating system isn’t the same as having it ready to go.

There’s still likely a lot of work to be done before the new rating system makes its public debut. The good news is that BDTi isn’t without friends.

Although BDTi will independently develop the experience rating, Qualcomm is just one of the companies that has already pledged support for the new rating, offering to provide both advice and insight into mobile device designs.

QTI believes in the need for a realistic, unbiased, and transparent benchmarking process that is designed to provide a reliable and accurate assessment of a mobile device's real-world capabilities. With an emphasis on measuring attributes relevant to actual user experience, BDTI's rating will avoid the pitfalls of current synthetic benchmarks and will provide more meaningful information to consumers.
Raj Talluri
Senior Vice President of Product Management at QTI

While QTI and BDTi’s claims sound great, you might be wondering how long before the rating system actually surfaces. The initial public device results should be released sometime later this year, with “general availability of the rating to follow in early 2014”. In other words, it’s all just around the corner.

No matter what, the most important factor for determining the right device will always be a combination of reviews, hype/marketing and just how well a particular device fits into your needs. Still, a mobile device score that reflects how well a particular device really works with Android?

If BDTi is able to pull this off, such a tool could really be a game-changer, giving folks a more realistic way to determine which device is ‘truly’ better, beyond the hype of a theoretical performance score.

Comments

  • Greg Cardall

    So they’re taking the “Windows Experience Index” and applying it to Smart Phones, adding Battery life in the mix. Even if copied from other ideas, I like to see someone taking mobile computing seriously, rather than creating yet another graphics benchmark program. I’d be curious to see if this takes off.

  • MasterMuffin

    Still half a year at least. I’m really interested in this, because S3 is still a really smooth device, but gets 15000 points less than top phones :/ Also, Moto X should perform better in this benchmark!

  • Ivan Budiutama

    depending on how neutral and how paid-proof the system, not to mention the fanboys and haters. The system will surely have a lot of challenge and it’s pretty difficult to actually measure “performance” with so many different variables there.

  • Ruz

    This is definately needed and i clearly tell u that Dual core 1.7ghz will easily outperform Quad core 1Ghz

  • Melad360

    plays by a different set of rules then your typical benchmark software.

    Should be than*, not then

  • SeraZR™

    you know you’re a noob when you believe in benchmarks :D