Chrome OS and Chromebooks are a perhaps the thinnest main-stream consumer client you can get today. Hell, next to Chrome OS, Android, IOS and even Windows RT, look positively fat with their full service OS functionality. No, make no mistake about it, Chrome OS is pretty much a browser in a box.
And even though most “real” laptop users will look down their nose at Chromebooks, these low-cost devices have consistently ranked on the top seller lists. In fact, we checked the Amazon top seller list for “laptops” and Chromebooks currently hold the top four out of five spots. Yes, we kid you not. So no matter what we think, it appears that Chromebooks are here to stay.
Since I’ve been a Chromebook user since the original Google CR-48, and have used almost every Chromebook ever made, I decided to see how the modern Chromebooks actually stacked up in performance.
Testing Chromebooks isn’t exactly easy though. Unlike Windows, Android or iOS, most of your “apps” are confined to the browser environment so stand-alone tests aren’t available today. Another problem I’ve come to realize with Chromebooks is that as Google pushes out newer versions of Chrome, the performance can be quite variable. Fortunately, we have in hand the new HP Chromebook 11, Acer’s new C720 as well as Google’s gorgeous Pixel, CR-48. We also fortunately still happen to have the Acer C710 to make this performance roundup. This let us update each of the available Chromebooks to their latest browser/webkit/V8 versions before running tests. One thing to note: only the HP Chromebook 11 and Acer C720 are running Chrome browser 30.0.1599.101 here. Google hadn’t pushed out the latest stable builds of the Chrome OS to its older Chromebooks at press time. Older Chromebook users shouldn’t fret too much though. It’s not like an Android handset that is exiled to an old OS. Our experience says Google does eventually update all of the Chromebooks to the latest COS because even the original CR-48 is just one version back now. One issue we have seen though with Google pushing out updates at seemingly random times: They can greatly impact performance by as much as 10 to 15 percent—in both directions depending on the benchmark. That means any meaningful benchmark has to be done basically in one shot and hopefully before Google pushes out a new update.
Chromebooks are also fairly unique from the Windows world. Generally with Windows laptops, old models are flushed out to make room for new units so they rarely overlap. With this roundup, we get to see the impact of various microarchitectures from Haswell to Sandy Bridge and Ivy Bridge as well as ARM since two of the Chromebooks here run Samsung Exynos 5 parts at 1.7GHz. We’ve noticed that Chromebooks is a sector where OEMs don’t seem to mind selling old units alongside newer units too.
For a sanity check, we also included the original Microsoft Surface Pro hybrid device. It features a 1.7-2.4GHz Core i5-3317U on the Ivy Bridge microarchitecture. Why no Surface Pro 2? We weren’t able to get our hands on one or we would have compared it to these Chromebooks as well. Rather than induce even more variability into it, we used the latest version the Chrome browser on the Surface Pro rather than Internet Explorer.
Intel Specs (click here for the complete side-by-side specs)
We let Intel do our work for us so you could eyeball the major x86 contenders in this Chrome-book-off. The Atom N455 is in the original CR-48, the Celeron 847 powers the Acer C710, the Celeron 2955U is in the Acer C720, and the Core i5-3337U runs the Google Pixel. Oh, the last chip, the Core i5-3317U is inside the Microsoft Surface Pro. Amazingly, the Acer C720 pulls off its performance win with a chip that has no Turbo Boost.
Best-sellling Amazon laptops (click the image for the full list)
We are not frakking with you—Chromebooks hold four of the top five spots on Amazon’s best seller list according to this screen cap we took Oct. 28.
Click on to page two for our benchmarks!
Chromeoff! (Click for larger image)
It’s no joke: 18.5 percent of compiled spec charts on web sites contain 3.1415 mistakes. See if you find out what’s wrong here.
Among the Chromebooks, it’s no surprise that the Ivy Bridge Core i5-based Google Pixel comes in on top. It’s almost twice as fast as the Acer C710 which uses a Sandy Bridge-based Celeron locked down at 1.1GHz. It’s also more than twice as fast as the two ARM-based Chromebooks, the HP Chromebook 11 and the Samsung Chromebook (that’s the official name.) If you look at our chart, the respectable performance comes from the Acer C720. It features a Haswell-based Celeron at 1.4GHz. It’s damn near as fast as the Google Pixel which is priced from $1,300 to $1,500. That ain’t bad. For Wintel folks, the Surface Pro’s numbers are pretty impressive. It’s faster than the Google Pixel on balanced and performance settings, but once the Surface Pro is set for Power Savings (something you may do to squeeze out more battery life) performance is actually on par with the ARM-based Chromebooks. The ARM-based Chrome have long been criticized as being a little under powered and we agree but it could be worse. The original single-core CR-48 platform shows how slow that old Atom chip was.
In FutureMark’s browser test we see an interesting result. The result is more graphics intensive as it tests HTML5 performance in graphics, physics and video among other tests. Even though the test seems to be quite different than Google’s Octane, the Chromebook performance results are almost identical to what we saw with Octane. The Pixel comes out on top, Surface Pro also represents well except on power savings mode where it’s as fast as the baseline Chromebooks. The one real change we see is the Acer C720 which turns in performance almost as fast as the Surface Pro and Pixel. Why? Our guess is the increase efficiency of the Haswell core vs. the older Ivy Bridge core powering both the Pixel and Surface Pro. We also think the graphics core in the Celeron Haswell could be responsible even though the Haswell Celeron’s features only “HD” graphics vs. the HD 4000 graphics in the Core i5 parts. We will note that the base clock of the Core i5 Ivy Bridge chips is 350MHz while the Celeron Haswell is 200MHz. One theory we’re floating is that the Haswell part may boost its graphics core for longer periods at higher clocks than the Core i5 chips can. The Core i5 chips can hit 1.05GHz and 1.1GHz while the Haswell Celeron can hit 1GHz. The older Sandy Bridge Celeron in the Acer C710 is so old it doesn’t even rate an “HD” from Intel. Its dynamic graphics frequency is also rated from 350MHz to 800MHz but we suspect it doesn’t run at 800MHz for very long.
Since most of the tests here run multiple tests and then calculate an overall score, we wanted to run something that might give us an idea of how well the Chromebooks would handle pure WebGL thrown at them. For that we used SpaceGoo’s WebGL Solar System experience which is compute heavy as it computes the hundreds or thousands of planetoids orbiting a larger planet. We start out with 1,000 planetoids and then let it slowly spool down. We report the frame rate when the planetoid count hits 900. We thought this would give us a better feel for compute power but our results have us questioning its value. The Pixel, which had outperformed all others, was barely ahead of the Samsung Chromebook and the Samsung Chromebook which internally is nearly identical to the HP Chromebook 11 was far faster. Surface Pro also easily destroyed all of the Chromebooks here. This makes us wonder if it’s a driver issue even as we expected the Surface Pro to be just slightly faster than the Google Pixel as both are built around similar chips. One thing we’re not surprised by is the CR-48’s performance because we’ve always known that thing was slow.
Wirple is a mostly HTML5 that measures browser performance in Canvas3D and WebGL and is heavily focused on 3D performance. There’s no surprises here as the Surface Pro and Pixel come out on top with the Acer C720 showing the best bang for the buck with it impressive performance given its $250 price tag. The ARM-based and older Sandy Bridge-based Celeron C710 offer pretty atrocious 3D performance comparatively.
Since few people have just one browser tab open, we decided to run the crowd through a multi-tasking environment consistent of an open tab to Google Music, Google Maps (set in satellite mode), Amazon.com as well as SpaceGoo’s WebGL planet and playing the 1080P version of the trailer for the movie Gravity on Youtube. With the trailer and WebGl actively running, we then ran the Kraken 1.1 benchmark. The results we saw showed that the slower Chromebooks had a far tougher time running so many tasks. We opted not to run it on the Samsung as its performance was pretty much the same as the Chromebook 11 and we also let the CR-48 go home early, because we know it’s already a dog. But to give you an idea how the Exynos 5 Chromebook 5 does, just peak back at the performance spread when it running just the Kraken benchmark along versus the multi-tasking load. One of the contributing factors besides clock speed and microarchitecture may simply be thread counts. The Surface Pro, Pixel and C720 all feature Hyper-Threading. The other Chromebooks were limited to but two threads and with so many workloads running, they were easily bogged down.
Our final test was a battery run down test. Rather than loop a video or just leave them sitting there, we decided to use the battery test in Futuremark’s Peace Keeper as our run down. This kept the WiFi on the notebooks hot and the screens on. The result? All of that performance on the Google Pixel (in addition to its high-res screen) meant pretty average to mediocre run time. Of the Chromebooks, only the Acer 710 was worse pretty much tying the Pixel at 195 minutes of use. The Chromebook 11 also performed pretty average with 241 minutes of run time. Frankly, we expected more. Its near twin, the Samsung Chromebook turned in a very usable 283 minutes. We’re not sure why the HP performed worse but it could be the IPS panel it uses versus the TN panel in the Samsung ARM Chromebook or even a smaller battery—though both publically state they have 30 Wh batteries. The real stunner is the Acer C720. More than performance, Intel has been chasing ARM on battery life. People know x86 kicks ARM ass in performance, but in battery performance ARM has the perception of awesomeness. That’s not the case here as the Acer C720 turns in a stellar 360 minutes of run time. That’s enough to use it on a cross-continental flight (provided you pay for inflight WiFi). Yay for x86 right? Well, yes and no. First, you can’t just compare one Chromebook to another and proclaim it a victory for the microarchitecture at hand. You have to look at all of the hardware around it and, well, the entire platform. This isn’t about ARM vs. x86 – it’s really about HP Chrome 11 vs. Acer C720. The battery in the C720, for example, is beefier than the C710. The Acer is almost half a pound heavier too—much of which we think comes from the bigger battery. What we can say is if you want stellar Chromebook battery life (something the anemic CR-48 was always known for) buy the C720. One final note on battery life: the Surface Pro really doesn’t do too badly. With Windows 8.1 set to power savings mode, the run time is but 38 minutes shy of the Chromebook 11. That’s not really as horrible as people think the Surface Pro will run. It’s also clear to us that the Surface Pro 2 with its Haswell part, should offer a nice upgrade in battery life over Surface Pro.
We do declare the Acer C720 as the current best Chromebook!
What’s the take away from this? If you’re shopping for a Chromebook, our recommendation is Acer’s new C720 based on performance and battery life. We do love the Pixel, but unless you’re rolling in dough, you probably can’t afford it. The price/performance/battery life ratio on the Pixel is also disappointing next to the Acer C720. Our only real beef with the C720 is its screen. It’s a pretty mediocre TN panel especially next to the IPS in the Chromebook 11. But screen isn’t everything. We did a simple test where we loaded up five major news sites on the C720 and Chromebook 11. As we know, media web sites tend to be loaded down with Flash and enough ads to make you think you were Las Vegas at 2 a.m. and simply scrolling down the web page with the Chromebook 11 would choke its ARM processor. It was enough to choke the C720 on occasion but the scrolling was acceptable. In our opinion, based on its price/performance/battery life ratio that makes the C720 the best Chromebook available today that we’ve seen.