There are quite a few nuances to that kind of folding, though...kind of intimidating for a "weekend" hobbyist. Component selection in that market is almost a joke. Newegg has one motherboard, and I'm sure there are more out there, but that's an indication of what you're looking at.
That motherboard, of course, doesn't seem to have a case available for it on newegg, so you'd probably be best served to fabricate your own. Then there's cooling and power, two considerations requiring careful planning.
I'd love to run hardware like that! That said, it ain't easy. Do all the homework you can, and there will still be wrinkles to figure out, since normal schleps like us aren't supposed to have that kind of compute power, according to Intel, amd, et al.
That said, it outperforms 10 $1000 gaming PCs by a pretty decent margin, for less than $5000. So, I definitely see the point. GPU/SMP folding is still probably the best the "weekenders" can do, though. For now, anyway.
Some good points and yes this is taking it slightly beyond "weekend hobbyist" so allow me to try and provide a bit more info.
You are right about newegg, at this level you need to be looking beyond "consumer" retailers - however there are several places where you can but a selection of M/B. Supermicro alone has 6 different models.
Case - this is a valid point, the boards are huge but there are a few that will either take a swtx board straight off or will do so with a bit of modding. Failing that spotswood does a very nice open bench case that will hold any m/b.
Power - any decent 850w or above will be fine for stock clocks, most overclocked boards run a 1200w PSU from what i've read. Cooling - there are G34 HSF available but they are designed for server racks so they ain't quiet. Musky over at [H] has come up with a simple mod to allow virtually any decent HSF to be used.
For anyone contemplating such a build there are guides out there to help but as ilander says its daunting to say the least, i've folded for nearly 4 years now and still haven't made my mid up whether or not to go down the 4p route.
Hope this helps a bit