Update: Be sure to check out our thoughts on what Microsoft MUST change for Windows 7
We sat down with Microsoft to hear the company’s side of the Vista story. What lessons have been learned following the worst
Windows launch in the company’s history? Is Microsoft doing enough to regain PC users’ faith?
Way back in January 2007, after years of hype and anticipation, Microsoft unveiled Windows Vista to a decidedly lukewarm reception by the PC community, IT pros, and tech journalists alike. Instead of a revolutionary next-generation OS that was chock-full of new features, the Windows community got an underwhelming rehash with very little going for it. Oh, and Vista was plagued with performance and incompatibility problems to boot.
Since then, the PC community has taken the idea that Vista is underwhelming and turned it into a mantra. We’ve all heard about Vista’s poor network transfer speeds, low frame rates in games, and driver issues—shoot, we’ve experienced the problems ourselves. But over the last 18 months, Vista has undergone myriad changes, including the release of Service Pack 1, making the OS worth a second look. It’s time we determine once and for all whether we should stick with XP for the next 18 months while we wait for Windows 7. But before we answer that question, let’s review exactly what’s wrong with Windows Vista.
We’ve seen worse launches over the years, but not from a multibillion dollar product that was a half-decade in the making. Here are the seven biggest contributors to Vista’s dud of a debut
At launch, we complained that Vista was significantly less stable than its predecessor. We experienced more hard locks, crashes, and blue screens in the first weeks of use than we had in the entire year prior using XP. Sadly for Microsoft, our experience was shared by many early Vista users.
The problems weren’t limited to high-end, bleeding-edge hardware, either. People with pedestrian, nonexotic hardware configs reported crashes, instability, and general wonkiness with Vista on laptops and desktops, in homebuilt rigs and OEM machines, and in PCs that originally shipped with XP. Considering that improved stability was one of the biggest promises Microsoft made for Vista, users were understandably upset.
Microsoft didn’t make any big promises about application compatibility, and it’s a damn good thing. If a desktop application didn’t follow Vista’s rules for behavior, Vista wouldn’t let it run. The program would fail to load, crash on use, or eat the user’s data, depending on the development infraction. And to be clear, we’re not talking about shareware apps created by some dude in his basement, we’re talking about Acrobat Reader, iTunes, Trillian, and dozens of other programs, not even counting the antivirus programs that are rarely compatible with a new OS.
Getting hardware working could be just as challenging. If you had one of the millions of perfectly serviceable, but suddenly incompatible printers or scanners, you probably felt pretty raw. We know we did.
Additionally, if you needed to connect to a VPN (virtual private network) that isn’t supported by Vista’s built-in client, you were probably out of luck. Vista shipped without support from major VPN manufacturers, including Cisco, leaving work-at-home types out in the cold.
The massive number of compatibility problems ensured that every user would be touched by at least one disappointment.
We would expect a new version of Windows to be slower than the previous one, given immature drivers and new features that drain CPU cycles and absorb memory. However, the performance differential has always been less than 10 percent in the past and only really evident in hardware-intensive apps, such as games.
At Vista’s launch, our tests revealed worse-than-expected performance in many different tasks and applications. Gaming performance suffered notably; using drivers from the launch time frame, our tests showed as much as a 20 percent performance difference between Vista and XP on the same machine. But that wasn’t the worst of it.
Even common tasks suffered. Large network file transfers took a ludicrous amount of time, even on systems hardwired to gigabit networks. On affected machines, Vista could take days to transfer a full gigabyte of data! While that was a worst-case scenario, many users complained that file transfers took twice as long to complete in Vista as in XP.
Vista brought marked improvements to the overall security of Windows, one of the few areas in which the OS actually lived up to Microsoft’s promises. Unfortunately, one of the mechanisms that helps enable that security comes at a high cost—it’s incredibly annoying.
That’s right, we’re talking about User Account Control, aka UAC. Even if you don’t know what it’s called, if you’ve used Vista, you’re undoubtedly aware that you need to prepare your clicking finger when the desktop darkens and your trusty PC starts asking whether you really meant to install that application you just double-clicked. UAC prompts you whenever an app tries to write to an area of your hard disk or registry that Windows finds suspicious. This seems like a good thing, right? It would be, except UAC prompts every time the installer does something suspicious. We’ve had Vista prompt us no fewer than five times before completing installs it questioned.
The problem is compounded by the fact that those five prompts look and behave differently, even though they’re all asking for basically the same thing: permission to write to a protected area of your system. To make matters even worse, none of the UAC prompts actually tells power users what the app is doing. When you click that Allow button, all you’re doing is adding a speed bump to whatever malware you might be installing.
Executed properly, UAC could have been a savior for people wont to install every application they find. Unfortunately, the UAC prompts quickly become so annoying that most users either disable them (the power-user option) or mindlessly click Allow (the mom option).
Activation has been a hassle since Microsoft first included it with Windows XP. Microsoft’s never really honored its stated 90-day limit for discarding activation information either. After installing the OS once or twice, you inevitably have to call some poor sap manning the activation hotline to enable Windows. What bothers us about Vista is the inclusion of the Windows Genuine Advantage software, which periodically checks in with Microsoft to ensure that the copy of Windows you’ve already activated remains genuine.
That’s all well and good, unless something confuses WGA. Unfortunately, just about everything confuses WGA. It could be something as simple as a BIOS reset that sets the clock back a few years. Or it could be that Microsoft’s entire activation process shuts down for a few hours—like it did last August. But at least Microsoft curbs piracy of Vista and other activated software by treating its customers like criminals, right? Well, not so much. Hacked versions of Vista that simply bypass activation are available on BitTorrent sites around the world.
In the old days, there were two distinct versions of Windows: one for home users and one for corporate users. For home, you bought Windows 98; IT departments bought Windows NT (at least the serious ones did). With Windows XP, this trend continued, despite the fact that both the home and enterprise OSes used the same core.
With Vista, the old home and enterprise distinctions went out the window, as Microsoft added three more versions of Windows, removing crucial features like the 3D UI from the low-end release and forcing power users who want access to both work-friendly and enthusiast features to shell out for the $400 Ultimate edition. To help justify that exorbitant price, Microsoft promised Ultimate Extras, the first of which didn’t materialize until months after launch, and those that did appear were disappointing. A bad Texas Hold ’Em game, a backup utility that should have been included in every box, and support for other languages do not “ultimate extras” make.
Oh, and if you used Windows XP Professional at home and wanted to upgrade to a less-expensive home version of Vista, you were out of luck. The only upgrade path that worked from XP Pro to anything with Media Center capability was the spendy Ultimate edition.
If the last eight years of watching Steve Jobs smugly introduce “one more thing” have taught us anything, it’s that no matter how technically sound (or alternately, how fatally flawed) a product is, every major release desperately needs one or two supersexy features to incite lust in geeks everywhere. Every time Jobs rolls out a new product, he teases the audience with a feature or two that you simply cannot wait to use. These features not only leave customers clamoring for the new product but also give those pesky users sitting on the fence a rationale for upgrading. While Vista had the technical chops in the form of the Aero renderer to deliver some potentially astounding apps, Microsoft’s best effort was Flip3D—a gimpy knock-off of a feature that OS X implemented infinitely better.
Aside from that, most of the apps included with Vista are rote updates of their forebears—from Movie Maker to Photo Gallery. There’s very little that’s new, even when the apps themselves are brand-new (see Windows Mail). Worse than nothing new, there’s not much in a stock Windows install to inspire anyone—even the stereotypical dullard PC user.
Vista was supposed to mark the launch of a new revolution in PC gaming, spearheaded by the full might of Microsoft as manifested in the Games for Windows initiative. With promises of everything from a fully fledged online matchmaking experience (a la Xbox Live), easier installations, and (most importantly) a host of killer AAA titles, Games for Windows looked poised to really challenge console dominance and modernize the PC as a gaming platform.
What Games for Windows actually did was tie the DirectX 10 API to Vista simply to drive sales of the OS. The first Vista-exclusive AAA Games for Windows title was a downright geriatric port of Halo 2, a game that originated with the first Xbox and doesn’t use DirectX 10! To add insult to injury, there was no technical reason for a three-year-old ported Xbox game to be Vista-only. True, the community quickly released a patch that opened the door for XP gamers, but we still can’t understand who possibly thought this was a good idea.
Microsoft continued down the suicidal Vista-only path for one more release, Shadowrun. Despite innovative gameplay and cross-platform support for its Xbox counterpart, the Vista-only release was enough to doom FASA Interactive, the studio that created the game.
We take a quantitative look at Vista and XP performance to determine exactly what penalty, if any, you pay when you upgrade to Windows Vista
To test Vista versus XP performance, we built what we think is a fairly middle-of-the-road rig—an Intel Q6600 quad core with 2GB of memory and a GeForce 8800 GTS videocard. We then ran a battery of benchmarks in three different OS environments: XP with Service Pack 3, Vista sans Service Pack 1 (with modern Nvidia drivers installed), and Vista with SP1. Our tests measure everything from overall system performance to network speed to gaming prowess.
Unsurprisingly, Windows XP remains faster in almost all of our standard system benchmarks. More noteworthy is how SP1 has improved Vista’s performance, narrowing the gap between that OS and XP in key tests and even allowing Vista to surpass XP in our MainConcept encoder test.
Unfortunately for Vista, our desktop benchmarks do reveal areas where Vista continues to suffer substantial performance hits compared to XP, namely in ProShow and Quake 4. We’ve talked to the ProShow developers, and they don’t know what causes the slowdown with their app in Vista, but they’re investigating. We attribute the Quake 4 performance hit to poor OpenGL drivers in Vista.
As we mentioned before, we’re perfectly willing to sacrifice a few percentage points of performance from an operating system upgrade. However, the difference between Vista SP1 and XP SP3 in ProShow and Quake 4 reaches a dismal 10 to 25 percent.
We didn’t include any DirectX 10 games in our tests simply because DirectX 10 wasn’t around when Vista launched, and DirectX 10 graphics still aren’t supported on Windows XP. Our basic system benchmarks already include a pair of games, FEAR and Quake 4, but we tossed in an additional round of 3DMark06 to further assess Vista’s gaming prowess.
The results were informative. Aside from the already noted Quake 4/OpenGL deficiency, Vista performed admirably both with and without SP1, turning in scores equivalent to XP’s. This tells us that the poor gaming performance we saw in the early days of Vista was more the result of immature drivers than issues with the OS. Of course, Microsoft can still be blamed for shoddy coordination with the graphics-card makers at the time of Vista’s launch.
Our final set of benchmarks test networking performance. We set up the fastest NAS box we’ve ever tested, the QNAP TS-109 Pro, and ran our standard network storage benchmarks on it. While we saw the same stunning performance inadequacies from pre-SP1 Vista that we observed at the OS’s launch, SP1 and the subsequent updates seem to have solved most of those issues. The minor gaps of a few seconds that do exist between XP and Vista SP1 are explained by the fact that XP shuts the file transfer window before the transfer is confirmed, while Vista waits until it has checked the copied file.
With the exception of a couple outlier applications, Vista’s performance is within striking distance of XP’s, for the most part. Thanks largely to a series of performance enhancements and SP1, Vista has closed the gap in many areas where it was deficient. We’re willing to overlook the poor OpenGL gaming performance simply because there aren’t very many OpenGL games coming out, and it seems the ProShow problem is an isolated incident.
|Windows XP (SP3)||Windows Vista (Launch)||Windows Vista (SP1)|
|Premiere Pro CS3 (sec)||924
|Photoshop CS3 (sec)||133
|Quake 4 (fps)||143.5||126.5||125.8|
|Windows XP (SP3)||Windows Vista (Launch)||Windows Vista (SP1)|
|3DMark06 Game 1 (fps)||29||28||28|
|3DMark06 Game 2 (fps)||26||26||26|
|Windows XP (SP3)||Windows Vista (Launch)||Windows Vista (SP1)|
|Network - Small to NAS (sec)||38||48||43|
|Network - Small from NAS (sec)||39
|Network - Large to NAS (sec)||139
|Network - Large from NAS (sec)||140
Abandoning the pretense that Vista is the perfect OS, Microsoft reps sat down with us to discuss the OS’s problems in a (kind of) frank conversation
We were surprised when Microsoft reps agreed to discuss Vista’s launch problems and what the company has done to fix them. We were surprised not only that they agreed to answer our questions with candor, but that they were speaking to us at all. Our initial conversation occurred in June and set the stage for the article you’re reading. This dialogue also marked the first time in eight years that we had a private conversation with any Microsoft employee without a PR manager present.
The answers we got during this mid-June background conversation were brutally honest: Our source, a high-ranking Windows product manager, conceded that Microsoft botched the Vista launch. He added that the company’s biggest concern wasn’t the OS but rather the eroded faith in Microsoft’s flagship product among users of all types and experience levels.
Our conversation was refreshingly frank, and no topic appeared off limits. To wit:
Yes, the June conversation was dazzlingly candid, and we were looking forward to an equally blunt follow-up meeting—a scheduled late-July on-the-record interview with Erik Lustig, a senior product manager responsible for Windows Fundamentals. But then the universe as we know it returned to normal, and Microsoft became Microsoft again. Our interview with Lustig was overseen by a PR representative and was filled with the type of carefully measured language that we’ve come to expect from Microsoft when discussing “challenges.” A “challenge” is Microsoftese for anything that isn’t going according to the company’s carefully choreographed plans. In the text that follows, we’ve combined the information conveyed during the mid-June background conversation with decoded translations of the “on the record” conversation we had in July. The contrast between the two interviews is stunning.
We herewith give you a snapshot of Microsoft’s take on Vista launch problems.
According to now-public internal Microsoft memos, 18 percent of all Vista crashes reported during the months immediately following its launch were due to unstable Nvidia graphics card drivers.
Microsoft has never issued any public comment concerning who’s to blame for the driver crashes, but during our background conversation, our source conceded that hardware OEMs were writing WDDM (Windows Display Driver Model) drivers for a moving target during Vista’s beta and release-candidate periods. Our source told us that because of low-level OS changes, hardware vendors didn’t have sufficient time to develop and test their drivers. This mirrors what Steven Sinofsky, the head of the Windows team, said in an interview with Cnet earlier this year: “The schedule challenges that we had, and the information disclosure weren’t consistent with the realities of the project, which made it all a much trickier end point when we got to general availability in January.”
Launch problems aside, once Vista is updated with SP1, it seems much more reliable than it was early on. The Maximum PC Lab isn’t equipped for long-term stability testing, but in our anecdotal experience, Vista’s stability problems are largely fixed, even on somewhat exotic hardware. Whether Vista is more stable than WinXP really depends on the actual hardware configuration you’re using more than anything else.
While discussing this story on background, Microsoft placed blame for incompatible software and hardware on its third-party partners. However, during our on-the-record chat, Lustig simply said, “I honestly don’t have the exact numbers for that,” in reference to the ratio of crashes attributed to Microsoft versus third-party entities.
Regardless, we’re well aware that Microsoft had been talking to hardware and software developers about Vista compatibility issues since the 2005 Meltdown, Microsoft’s annual gaming conference. At that conference, Microsoft informed game developers that they needed to write apps that behaved well, or they would face problems with Vista. The requirements were, for the most part, simple—caveats like not writing to C:/Program Files/ or C:/Windows/.
It’s also important to note a shameful truth that everyone in the PC industry is aware of but rarely discusses: When a new OS comes out, third-party vendors will often withhold compatibility support in order to drive sales of new units, turning the cost of supporting a new OS from a liability into a source of revenue. The same goes for software like antivirus utilities and some CD/DVD burning apps, both of which hook into the OS very closely.
The statistics on Vista’s security record are clear: Vista is the most secure version of Windows to date. Nonetheless, Lustig said that Microsoft made “changes that have had some short-term ramifications that we’ve worked very hard the last year and a half, and through Service Pack 1, to address.” Some of these changes may have had unintended negative consequences, but Vista has suffered fewer security defects than any previous version of Windows. In short, sometimes you just have to give up flexibility for security. As Lustig told us, “I believe that those changes are going to be a fundamental basis for the integrity of the platform.” We agree.
During our initial June interview, Microsoft blamed unoptimized videocard drivers for poor gaming performance. To confirm this, we tested both the launch version of Vista and the post-SP1 version of Vista with current Nvidia drivers. Our gaming tests showed only the most negligible performance differences between the two OS builds, confirming that Vista itself was not to blame for early game performance issues. Rather, those earliest Vista videocard drivers were the culprits. Indeed, now 18 months after its launch, Vista’s performance is within striking distance of WinXP’s in almost every test we ran.
Because Vista’s first Service Pack significantly improved the struggling OS, we were surprised that Microsoft didn’t tack a Second Edition label on it, a la Windows 98. Providing measurable improvements in performance and stability, Service Pack 1 should have been Vista’s saving grace. No? Lustig told us that despite significant improvements in most of Vista’s deficient areas, “there is a lot of leftover concern [about Vista] based on information folks have heard anecdotally.”
Quite an admission. Lustig continued, “The challenge for Microsoft isn’t necessarily continuing to take the feedback and improving the product—we’ve been doing that since launch and will continue to. The challenge is getting the message out that we’ve listened, we’ve made very positive changes, we’re seeing very positive results from the changes we’ve made, and there’s enough value in the product.”
After spending the last six weeks getting down and dirty with the OS—on multiple hardware configurations, in both 64-bit and 32-bit flavors, and on mobile and desktop systems—we’re willing to give it a second chance. There are still tons of things about the OS we’re not happy with—starting with the now-$350 Ultimate SKU and working down from there—but from a performance, stability, and security standpoint, we’re satisfied with where Vista is today. You no longer need to sacrifice performance or stability if you want to run the latest version of Windows.
If you already have Vista, there’s no reason not to use it, but should you go out and buy Vista today? Probably not. With Windows 7’s launch scheduled for early 2010, we’re actually closer to that date than we are to Vista’s launch. If you’ve ridden out the storm on XP so far, it probably isn’t worth investing in Vista for just a year and a half of use.