Factor these (now) thirty-six tests against an average of ten test suite iterations--a minimum number of variances that Resig runs in a common jQuery testing environment. That's three hundred and sixty runs for every test you create, more if you're expanding to include OSX and Linux platforms. And did I mention that the best results tend to occur when actual human beings are behind the testing instead of some automated attempt at user interaction? Yeaaaah...
To a user, the TestSwarm client just simply works. When you load up the page, the program checks to see what browser you're using and determines whether it's one that is needed for a round of tests. If so, TestSwarm pops up a little window and asks you for your help. If you choose to enter a username and agree to join the fun, you're placed in a holding queue. The TestSwarm client checks for new tests to run on your machine over a set interval of time. If a particular test screws up, a detailed note is sent back to TestSwarm to help developers identify the root cause of the issue. They also receive a giant color-coded chart that shows off the different tests and browser permutations, as well as a visual representation of tests that succeeded, succumbed to minor errors, or completely fell apart.
It's an awesome idea--at least, I think so. But what do users get for their contributions to either TestSwarm or the now-aptly named TestMurph?
Sure, Resig could turn TestSwarm into a downloadable application that launches browser windows during your computer's idle time. But that's a pretty hefty amount of code spread across multiple operating systems, not to mention an increased amount of steps and potential annoyances for users looking to help out. It's a catch-22 if I ever heard one: To increase TestSwarm's popularity and applicability, one has to increase the program's complexity and user interactivity. But unless TestSwarm is exposed to as many permutations of browsers, operating systems, and setups as possible, the entire point of the platform dies away.
To his credit, Resig has opened the doors with an innovative idea for online testing that's sure to be replicated, modified, and distributed in the days to come. In fact, there's been a bit of interest in corporate versions of TestSuite, which bodes well for future TestSwarm spin-offs. I just hope that, for all his work and creativity, the single variable out of Resig's control doesn't ultimately prove to be the suite's undoing. Were there only some equally innovative way to encourage the adoption of the experiment by its chief guinea pigs--that's the real question here.
David Murphy (@ Acererak) is a technology journalist and former Maximum PC editor. He writes weekly columns about the wide world of open-source as well as weekly roundups of awesome, freebie software. Befriend him on Twitter, especially if you have an awesome app or game you're dying to recommend!