Print
Happy day-after-Firefox-release day. If you're one of the 3.2 million Americans to download the latest release of the browser as of this column's writing, congratulations. You, like your peers, have recognized the value of upgrading to faster and better technology products! If that sounds weird, that's the point. It should. According to Net Applications, around twenty percent of users (out of a survey sample of around 160 million people) still use an older version of a Web browser, be it Internet Explorer 6, Firefox 2, or either Safari 3.1 or 3.2. You are not among them; I salute thee.
You've probably read a lot of marketing in the last 24 hours about how fast, awesome, and packed-full of features the new Firefox 3.5 release is. Since you've had a chance to play with the release candidate of this latest upgrade starting in early June, this shouldn't come as much of a surprise. But let's cut through the press release and examine the real facts: Just how much faster is Firefox 3.5 over its browser brethren? Has Mozilla's newest TraceMonkey JavaScript engine delivered a princess or a barrel?
Yes and no.
If we're just considering a Firefox-only universe then, yes, the browser's performance is quite an improvement over its predecessor 3.0.11 release. You can partially thank Adobe for that. Mozilla interwove the company's just-in-time compiler nanojit, released as open-source in 2006, alongside a new tracing system to create Firefox's new JavaScript engine. Without getting too technical, the tracing engine streamlines Firefox's operations by recording the path that frequently accessed JavaScript code takes through an interpreter. It then compiles this trace into native code, which can be called up and duplicated faster than passing the code through the interpreter once again.
The industry-standard SunSpider JavaScript benchmark attempts to highlight differences in browser performance by running through a series of real-world use patterns. More than that, the program runs through enough iterations of the tests to calculate a measurement of the run's statistical significance--a determination of the accuracy of your results and their validity for real-world comparisons. Using this very benchmark, Harry McCracken of Technologizer notes that Firefox 3.5 delivers a performance improvement that's 2.4 times faster than Firefox 3.0.11. But Google's Chrome 2.0 beta takes the cyber-cake in the end, just squeaking by Mozilla's masterpiece on the benchmark charts.
Samara Lynn from ChannelWeb ran her own SunSpider browser evaluation, sticking to Google Chrome, Firefox 3.5, and Internet Explorer 8. Her numbers gave Chrome an advantage of nearly 600 milliseconds, or a 39 percent decrease in time from Firefox 3.5 to Chrome. Internet Explorer 8 sank to the bottom of the listing as if it had a rock tied around its status bar, delivering a time of 8,131.8 milliseconds to Chrome's 924.2 (lower is better).
So who's right? Lynn? McCracken? TGDaily, which puts Chrome's time at 628.4 milliseconds (a 48 percent decrease compared to Firefox 3.5)? Nobody and everybody. While the rankings between the browsers remain the same within these three sites, as well as my own personal comparisons of Firefox and Chrome in SunSpider, the numbers vary depending on the system setup. That makes it a little difficult to decide the close races, especially since TGDaily has Chrome beating out Safari by roughly 60 milliseconds. At least we can all agree that Chrome is faster than Firefox 3.5, right?
A similar situation occurs on Futuremark's Peacekeeper browser benchmark. Although TGDaily claims that Chrome won't run it, both Lynn and I received scores when running the program through Google's browser. She has Chrome beating out Firefox 3.5 by a score of 2747 to 1843, a 49 percent speed increase from Firefox 3.5 to Chrome. I found a 55 percent increase in performance on my own benchmark run, with Chrome overtaking Firefox 3.5 to the tune of 3,073 to 1,978. Just for the sake of a good joke, Lynn pegs Internet Explorer's performance on this test at a whopping 675. That's not even half of her recorded score for Firefox.
Just to throw one more benchmark in for good measure--because I don't exactly trust Google's Chrome V8 benchmark that suggests Chrome is nine times faster than Firefox 3.5--TG Daily ran one of my favorite evaluations that tests Flash performance in a browser. The run, called Le Crabe, measures how many individual animations your screen can hold before the frames-per-second score dips below a particular amount (25). On this, Firefox 3.5 crushed the competition, holding out for 636 total crabs on TG Daily's setup to Google Chrome's 241. Stranger still, even Internet Explorer itself pulled out of dead last to deliver an impressive second-place performance amongst Firefox 3.5, Safari, and Chrome.
You made it this far--so which browser is faster? Eh. To really get a sense of how your browser performs, you have to factor in more than just the JavaScript benchmark numbers. What's the memory use of the browser? What kind of content exists on the sites are you hitting up? What's your Internet connection? While these benchmarking tools have allows us to legitimize the differences between Firefox's versions on a functional level, and help highlight the various browsers' abilities in certain areas of rendering, there's no clear-cut winner based on the numbers. After all, it's difficult to weigh certain performance aspects over others. And as you've seen, individual performance characteristics can vary greatly depending on the testing platform. Browser benchmarks are great for comparing version performance--for a big-picture guide, there's just so much more to consider.
Speed up your geek involvement by befriending David Murphy @acererak. He's three-point-five times as fast a twitterer as any other geek, save perhaps Nathan.
Comments are closed on this article
ZayLay
July 02, 2009 at 7:05am
Java Script Benchmarks alone is like the smallest fraction of the internet browsing experience. How quick does flash load, flash video play, image loads, long table pages, heavy div layers, multiple tabs, applets, processor usage, memory usage. Where are the other benchmarks besides java script.
ironmon1
July 01, 2009 at 3:29pm
raw speed tests are great, but they do no necessarily represent the real-internet performance of the browsers-I, like many, run firefox with an array of add-ons. Some add-ons, like adblock, are not availible for chrome, and help web pages load faster by not bothering to load ads.
TheMurph
July 01, 2009 at 3:58pm
I agree. I agree so much that I will recopy part of the last paragraph in the article for your perusal:
"You made it this far--so which browser is faster? Eh. To really get a sense of how your browser performs, you have to factor in more than just the JavaScript benchmark numbers. What's the memory use of the browser? What kind of content exists on the sites are you hitting up? What's your Internet connection?"
SeVeN
July 01, 2009 at 3:15pm
David, thanks for the review. But I wonder if the benchmarks were done with Firefox tuned/tweaked? Becuase I've done some benchmarking with Firefox 3.5 tweaked (using firetune), and the results were totally different in favor of Firefox. It was about 55% faster than Chrome. I don't know if Chrome can be tweaked as well, but it'd be only a fare comparison if both browsers were benchmarked at their maximum strength. Just wanted to share that with you, and it'd be good to get some feedback.
TheMurph
July 01, 2009 at 3:53pm
Interesting. I haven't futzed around with firetune myself, but would be curious to hear about the different settings you used to maximize your browser's performance.
As for the benchmarks, I can only assume that they were run on plain-jane browsers. No add-ons, mods, whatever. That's how I ran mine, at least. One has to think of the normal users, after all--I bet there are people out there who have no idea what a Firefox add-on even is.
demosthenes
July 01, 2009 at 1:05pm
So, chrome is faster, I get it, and by a margin measured in the same units usually reserved for things like blinking. I like google, I even own stock; so as soon as they add support for extensions we'll be good to go. Don't get me wrong, faster is better, even if it's just a tiny bit. No extensions is a deal breaker, hands down, end of story. I just can't see giving up all that cutomization for a very small speed increase.
____________________
.a nut for a jar of tunA
TheMurph
July 01, 2009 at 1:15pm
One can say the same thing about any technology: what separates a videocard that spits out 70 frames per second versus a videocard that spits out 75? Can you actually see a discernable difference?
Although a number of other factors should go into your purchasing decision beyond mere speed ratings--the same holds true for the browsers you download--some people nevertheless want the fastest technology they can get, regardless of any external factors. This article thus served a twofold purpse: Answering whether Firefox 3.5 is faster than 3.1 (and to what extent), and whether the improvements in Firefox 3.5 are enough to topple the other browsers on the benchmarks.
