System Performance
Section by Brandon Chester
The Pixel XL's SoC is designed by Qualcomm and comes with four of its custom Kryo CPU cores—two with a peak frequency of 2.15GHz and two more that top out at 1.59GHz. The quad-core CPU is paired with an Adreno 530 GPU. After hearing that, you may think that I'm describing Snapdragon 820, but in this case I'm actually describing Snapdragon 821. Qualcomm's SoCs often come in two versions with different sets of frequencies, and in the case of the Pixel XL Google has opted to use MSM8996 Pro-AB, which shares the same clock speeds as the normal MSM8996 that shipped in many of 2016's flagship smartphones.
On paper, there's not really any room for Google to have improved general CPU performance compared to the other Android flagships from this year. However, the Pixel XL comes with Android Nougat, along with Google's firmware. System performance on a smartphone is hardly just a function of how fast the CPU is, and it's generally not something that you can determine based solely on the hardware, even if you consider all the parts of the system. A phone's software plays a big role in its UI performance, as well as the performance of critical Android APIs. We've seen devices with the same SoC achieve very different results in tests such as PCMark that utilize frameworks such as RenderScript and core APIs like the system WebView. In order to get a better idea of how well the Pixel XL performs, I've run it through our standard set of benchmarks.
Like the Nexus 5X and 6P, the Pixel XL performs very poorly in PCMark. The overall score actually comes in below the Nexus 6P despite the Pixel XL being a much faster device on paper. The Web Browsing score is much lower, which is shocking when one considers that improvements to Android's native WebView in Marshmallow put the Nexus 6P at the top of the chart in that test. Video Playback sees a slight reduction, and the Writing score is in line with the Nexus 6P, which performs exceptionally poor in that test. The Photo Editing test, which utilizes some GPU compute, is the only area where performance improves thanks to the Adreno 530 GPU, but it's not near enough to offset the poor performance in the other tests. Devices that use Snapdragon 820 or Kirin 950 outperform the Pixel XL by a large margin, and it actually sits among devices using SoCs from 2015 like the LG G4, the Galaxy S6, and the previously mentioned Nexus 5X and 6P.
Google should really be investigating why their performance is so poor in PCMark for Android. All of the tests are perfectly legitimate scenarios that reflect real-world interactions, and these performance problems will be replicated in real apps that perform these tasks using similar code. For example, the web test uses Android's built in WebView, while the writing test manipulates text and does file IO using Android's own Java APIs. The performance issues when executing tests written purely in Java suggests that there are software problems that may trace back to the Android RunTime, and it's concerning that such problems can exist when building software that mainly uses Android's own APIs.
Storage Performance
[Update 11/9/2016: Because AndroBench 4 is not compatible with Android 7, we were unable to include storage performance results initially. After our testing was complete, AndroBench 5 was released, which resolved the compatibility issue. However, this release escaped our attention, because we were busy finishing the review. Now that we've had time to try the updated version, we've gone back and tested the Pixel XL and several other phones.]
In the updated AndroBench 5 storage performance test, our 32GB Pixel XL unit consistently outperforms our 128GB unit by a small amount, although performance for both is not great. Sequential read performance is average, coming in 33% slower than the Galaxy S7 edge. Both Pixel XL units trail the Nexus 6P in the sequential write and random read tests, and they also fall behind the few other phones tested so far.
These results align with our subjective experience: File operations on the Pixel XL are noticeably slower than on some other recent phones. In some cases the difference is small to negligible and only noticeable if compared side by side. The Honor 8, however, is much quicker than the Pixel XL when launching/installing apps or reading/writing files.
Browser Performance
While the performance in PCMark's web test using Android's native WebView is nothing to get excited over, the performance in Chrome on the Pixel XL shows much more promising results. The Pixel XL ships with Chrome v54 out of the box, which promises improved performance. In Kraken and WebXPRT 2015 Google achieves the best results of any Android device, and in Jetstream only Huawei's Kirin 950/955 devices with their Cortex A72 cores are able to best it. However, when looking at the overall results it's clear that Google is still lagging far behind the web performance of Apple's mobile devices. It's actually concerning that in the grand scheme of things, the only praise you can really give the Pixel XL is that Google has finally beaten the iPhone 6 in web performance, even though the iPhone 6 is over two years old.
The general CPU and system performance of the Pixel XL is something of a mixed bag. Google has done work to improve on Chrome's poor JavaScript performance on Android devices, but there's still a long way to go. PCMark also highlights the fact that there are odd issues with general performance, even in cases where the code being executed is purely Java code using Android's own APIs. These issues existed on the Nexus 5X and 6P, so it doesn't look like Google has spent any time trying to figure out what the cause is. In addition, the score in the web test which uses Android's native WebView is actually significantly lower than the Nexus 6P, and it's never good to see performance regress.
Of course, none of this really speaks to the Pixel XL's UI performance, which is exceptional. Google has clearly put effort into reducing jank and optimizing the performance of application switching. While it's hard to measure UI fluidity, it's obvious that some devices are better than others, and that's very true of the Pixel XL. As for app loading and switching, we would normally use DiscoMark to measure this. Unfortunately, this is no longer possible as of Android Nougat. Changes that Google have made to the operating system mean that observing accessibility events on the system will not give you an accurate idea of how long it took for an application to load. I initially noticed this when I realized that the Pixel XL was logging as being four times faster when launching apps after a reboot compared to apps already in RAM, which obviously doesn't make any sense. We reached out to the developers of DiscoMark, and they confirmed that this is a known issue.
Ultimately, it's a bit unfortunate that there are still obvious gaps between the performance of Google's best Android device and an equivalently priced iPhone. Tasks like web browsing and working with large groups of text or other data are just simply faster on the iPhone 7 than the Pixel XL. The fact that there are improvements to web performance compared to competing devices is a good sign, and I hope we'll hear more about efforts to improve performance in these areas as time goes on. I think users will greatly appreciate the work that Google has done to optimize UI performance and the loading of applications, as it has a substantial impact on your perception of how fast the phone feels. I'm also hopeful that we'll be able to quantify the lead that Google has here compared to other devices sometime soon.
ncG1vNJzZmivp6x7orrAp5utnZOde6S7zGiqoaenZH5xg5RsZq2glWK0sLvGpZxmqJmtsq1516Vkq52mnrK4e5Q%3D