Tested: Snapdragon-powered Android phones just got a lot faster at ray tracing

Smartphone ray tracing is here. Well, sort of. We don’t have any actual mobile games with ray tracing support yet; we’re still waiting patiently. But we do have Android phones with hardware ray tracing capabilities and benchmarks to test these graphics number crunchers.

The popular 3DMark benchmarking suite has just received its first ray tracing test — Solar Bay. We’ve grabbed the app and three flagship smartphones packing the currently available mobile ray-tracing chipsets — the Snapdragon 8 Gen 2, Dimensity 9200, and Exynos 2200 — to test it out and see where the handsets stack up.

What do ray tracing benchmarks test?

Ray tracing can enhance a selection of graphical techniques, most commonly lighting, shadows, and reflections. As such, game and benchmark developers can choose how to use ray tracing to balance visual fidelity against performance costs by using ray tracing in different areas. Our first ray tracing benchmarking session, using Basemark’s InVitro test, looked exclusively at reflections, with traditional rasterization used for lighting and other effects.

A dive into UL’s documentation reveals that 3DMark’s new Solar Bay software features ray-traced reflections, volumetric lighting, particles, and post-processing effects, making it a more expansive test. The benchmark is also split into three sections, where the second and third sections target an average of 2x and 3x the ray tracing workload of the first. This gives us a good idea of how GPUs scale with varying ray tracing workloads.

While it might be tempting to compare 3DMark to Basemark, the scores won’t be too compatible as they test slightly different things. Still, both offer a good look at how mobile GPUs will perform with real ray tracing games (when they finally arrive), as titles will undoubtedly vary from hybrid rendering like InVitro to more full-blown ray tracing implementations like Solar Bay. Let’s see how the results stack up.

3DMark Solar Bay ray tracing benchmark results

First, a single run of Solar Bay on our Samsung Galaxy S23 Ultra (Snapdragon 8 Gen 2), Samsung Galaxy S22 Ultra (Exynos 2200 model), and vivo X90 Pro (Dimensity 9200), with hardware ray tracing support on their Adreno 740, Xclipse 920, and Immortalis-G715 GPUs, respectively.

There are a few key points to digest here. First, the Snapdragon’s Adreno 740 and Dimensity’s Immortalis-G715 top the charts, with the latter winning by a roughly 10% margin on average. The older Xclipse 920 GPU in the Exynos 2200 doesn’t perform anywhere near as well, clocking in 33% lower frame rates than the leader. This isn’t surprising, given that the older chip’s GPU shows a large performance deficiency in classic graphics benchmarks too.

The Exynos also has the more variable of the three GPUs, showcasing a larger performance drop as the level of ray tracing processing increases. Arm’s Immortalis GPU is very impressive by comparison, with performance dipping by less than 5% when doubling the ray tracing workload. This helps the chip top the overall score, but it’s virtually tied with the Snapdragon in the first stage’s lighter workload. This suggests that the chip’s ray racing hardware has a decent level of headroom before bottlenecking the GPU, while the Adreno and Xclipse GPUs struggle more as the level of ray tracing calculations increase.

With the Galaxy S23 Ultra performing very well here, we sensed a driver update, so we re-ran our early 2023 InVitro tests, which initially showed sluggish ray tracing performance for Snapdragon-powered smartphones.

Our suspicions appear confirmed; a graphics driver update or similar has been applied to the Galaxy S23 Ultra since our initial run. Other chipset results are essentially unchanged. Average frame rates jumped from a peak of 18.6fps up to 27.1fps, a colossal 45% gain that puts the latest Snapdragon 8 Gen 2 driver firmly ahead of the Exynos 2200 and much closer to the Dimensity 9200. These results are much more in line with results from 3DMark’s benchmark too.

After installing several firmware and Google system updates, the ASUS ROG Phone 7 also showcases a similarly high 26.5fps, as does the Sony Xperia 1 V out of the box. However, we haven’t clocked the uplift on every 8 Gen 2 phone in our possession, so this update may still be making its way around, or it could be because some are review units. Either way, it’s good news for gamers eagerly waiting for the first ray tracing titles this year.

Sustained ray tracing performance testing

Of course, a short benchmark is one thing but not very reflective of a longer gaming session. For this, we turned to Solar Bay’s stress test mode of 20 consecutive runs to see how these chips perform once temperatures start rising.

We already knew that the vivo X90 Pro and its Dimensity 9200 chipset run hot, which is apparent in the above stress test. The chip gives up its lead only after a few runs, falling behind the more consistent Snapdragon 8 Gen 2. The Exynos chip is equally inconsistent, gradually reducing performance as temperatures increase.

Interestingly, we see more severe throttling from all of the chips in the 3DMark stress test compared to Basemark’s. While we shouldn’t compare these quite different benchmarks too closely, this suggests that either the GPUs are more bottlenecked in InVitro, which seems unlikely given the higher frame rates versus Solar Bay, or that 3DMark’s ray tracing suite is stressing the GPUs harder. This seems the most plausible, owing to the more extensive use of ray tracing throughout the scene. This is illuminating, as developers will want to hold off throwing too many ray tracing effects at first-gen mobile ray tracing GPUs to ensure higher frame rates and better-sustained performance for long gaming sessions.

Ray tracing is ready for prime time

Of course, benchmarks aren’t a drop-in replacement for real-world performance, and although we typically expect them to be more demanding, ray tracing performance for real games remains a bit of an unknowable until the first titles land in our hands, hopefully later in 2023. Will games be able to hit lofty expectations of 60fps or will they be stuck struggling for a steady 30fps? We’ll have to see.

While there are clearly limits on the level of ray tracing calculations we can throw at mobile devices, two of three mobile chipsets that currently support hardware raytracing seem able to handle not just ray traced reflections but lighting, particles, and post-processing too at close to playable frame rates. But perhaps the most significant news here is that Snapdragon 8 Gen 2 phones, which we initially worried would be pretty sluggish, are right near the top of current ray tracing performance metrics following an update.

With next-gen mobile GPUs heading our way with the Snapdragon 8 Gen 3, Tensor G3, and others, ray tracing will soon be a staple of high-end mobile gaming, and it’s shaping up pretty alright.