Open Up to Open Formats

Like pretty much all of you who read this site, I spend most of my working day messing with data and text files, and sending them around to various colleagues on a variety of platforms. While I do most of my work in OpenOffice.org and The GIMP, the overwhelming majority of my colleagues use Microsoft Office, Adobe Photoshop, and Adobe InDesign. So, despite my personal preference for Free and Open Source Software and open-standard documents, almost every document I create eventually ends up in some proprietary format. But proprietary formats, while convenient in their ubiquity, are inherently problematic, and there are some very compelling reasons to think twice about how you save your files. Here's why you should use open formats, and convince your colleagues (hint, hint to my colleagues) to do the same.

Vendor Lock-In
Proprietary formats, such as DOC and MP3, are dangerously common. While almost everyone on the planet uses these formats to save and share their data, the formats themselves are not in the public domain, but belong to corporations (in these cases, Microsoft and Thomson Consumer Electronics (et al.), respectively. Admittedly, this doesn't usually pose much of a problem in the short term. Songs you ripped to MP3 ten years ago still play just fine on your current player, and you can still open Word docs you created in the early '90s. The trouble comes when you – or the company that owns the format – decides it's time for a change.

Case in point: This year, Microsoft has decided to introduce a new format, Open XML (aka DOCX) which it has declared the new standard for Word files. Immediately, a schism broke open. While almost everyone on earth is still using the older DOC format, many who upgrade to Office 2007 unwittingly began using the new format (set as the default for Word 2007), which is unreadable to their colleagues without the use of a conversion utility. Time spent fiddling around converting file formats means lost productivity and lost revenues for businesses, governments, and end users who didn't think there was anything particularly wrong with the old format.

Alternatively, businesses, governments, and end users sometimes decide of their own accord that it's time for a change. Maybe they decide to quit spending their money on exorbitant licensing fees, or perhaps they simply find a different application more suitable to their needs. Whatever the case, the decision to abandon one product in favor of another shouldn't leave a company's assets (the terabytes of data that represent its most valued intellectual property) at the mercy of the vendor they've left behind. But all too often, what begins as a simple business decision ends up a data catastrophe, thanks in large part to the arbitrary intricacies of proprietary formats.

In my days as a net admin, I saw a company nearly screech to a halt in its attempt to flee an obsolete accounting system after the original vendor closed its doors. Tens of thousands of dollars in consulting and outsourcing services saved the company from a gruesome fate (and ensured that I got my next paycheck on time). If the company had been able to use an open format for its precious accounting files, rather than a closed propriety one, that transition would have been relatively simple and inexpensive. Most real-world scenarios are less dramatic than that. After all, anyone who opts to switch from Microsoft Office to OpenOffice.org (or the other way 'round) is unlikely to find themselves up a creek, since both suites support a wide variety of current and legacy file formats. But in principle, the concern remains.

---

Proprietary formats thrive not so much because they're superior as because they encourage an atmosphere of fear, uncertainty, and doubt among end users. If someone's been using Microsoft Word for a decade, and all of their old files are in the DOC format, they'll most likely feel (whether it's reasonable or not) that they should continue to buy upgrades to Word in perpetuity, out of an ill-founded concern for backward compatibility. The reality, however, is that proprietary standards create greater compatibility problems than open ones, because they are less subject to approval and oversight, and the changes they undergo tend to serve the interests of the vendor rather than the end user.

Over the past several years, we've seen an explosion of proprietary standards in the music industry, all in the name of digital rights management. Rather than increasing choice and opportunities for consumers, DRM-laden music formats place artificial limitations on the number of devices (and types of devices) we can enjoy our music on. As a point of comparison, you can buy a CD at the store and play it on any CD-playing device you find, whether it's in your car, your computer, your portable player, or your alarm clock. The disc doesn't care. and you've paid for the right to play it anywhere. (And to make a backup copy, if you so choose, or rip it to a digital format that you can play on your iPod.) DRM whittles down that choice by limiting you, in most cases, to three devices per track (and typically offering poor or limited options for de-authorizing devices). Meanwhile, many proprietary formats are of lower quality than their open-standard counterparts, even if they're not riddled with DRM code.

Open Standards
The good news is that the digital world is brimming with reliable open standards and formats, many of which already enjoy more widespread use than their proprietary rivals. One such example is PDF, which was developed by Adobe but is used almost universally, and will likely soon become an official ISO international standard. Other great examples include HTML and XML, which together dominate the interwebs.

But there are less known open formats out there that should get more attention than they do, but languish in comparison to more established proprietary formats. Chief among these is ODF (aka OASIS OpenDocument Format). ODF is the default file format for OpenOffice.org, and it's an internationally accepted ISO standard. Like Microsoft's new Office Open XML (DOCX) format, ODF is based on XML, which makes it extremely flexible for various kinds of files, from spreadsheets to text documents and graphics. However, unlike DOCX, ODF was developed by an international consortium, rather than a single company, and is based on existing standards. Microsoft's OOXML, on the other hand, is a wild departure from existing standards that will almost certainly cause more interoperability headaches than it will solve. What Microsoft's real intention may be in the development of OOXML remains to be seen, but it likely has more to do with posturing for future patent suits than with serving the public interest.

For music, the OGG media standard is one of the best around, and its popularity is growing fast. It's flexible, open, and it retains excellent sound quality. Unfortunately, some leading media players don't support it, because their manufacturers have a vested interest in saddling you with DRM-laden files of their own design. But if you really care about music, skip the tip to the online music store and buy real CDs. Then you can rip them as high-quality, lossless OGG-FLACs that you'll own forever.

UPDATE (7/11/07 7:32PM PDT): For more specific analysis of Microsoft so-called Open XML (an ironic name, since XML is already an open standard), check out this critical post by Rob Weir. (Via /.)