Okay, that's a valid point, but if all the hardware is connected, that still doesn't tell you anything other than there's something wrong with system, because again, you have all that hardware connected and any one of them could be at fault.
Two ways to diagnose a problem. First means using basic electrical concepts to identify a problem before anything is disconnected. That is how the informed do it. And summarized by a sound byte "Work smarter; not harder".
Second technique is done by many who do not learn how hardware works. Also called shotgunning. Keep replacing good parts until something works.
Easy is to see a failure before removing any component or disconnecting any wire. But that means learning good diagnostic procedures. Therefore problems are solved faster and without spending money replacing good parts.
Using a multimeter on a power 'system' (which is more than a PSU) means a problem can be identified immediately. But again, that means learning techniques we were using long before the IBM PC existed. IOW "Work smarter". Many have no idea how to quickly identify power system failures with a three digit multimeter.
Identify the entire power 'system' as good or suspect. A meter does not just exonerate or acccuse a PSU. It can verify integrity of the entire power 'system'. Accomplished in a minute with a meter. And yes, the meter can clearly define the power system (including PSU) as functional or defective ... without any doubt or speculation. But that means no component is removed and no wire is disconnected. An example of how to "work smarter". As we did routinely even before an IBM PC existed.
Removing a PSU means some defects in that PSU cannot be identified (unless one has $thousands in test equipment).
BTW nobody has said anything about reparing a PSU or even opening its case. We sometimes do that to identify why that component of a power system has failed. But then we do things that require higher reliability. What is a most common reason for failures? Manufacturing defects. But analysis at that level is unnecessary for the OP.
chuckabt also noted
When I sent the board back to Gigabyte they stated on their site "bad component" and replaced it.
That does not mean a motherboard was tested. Better manufacturers simply declared rumored moterhbard (also called a component) as defective. And automatically replace it. Some are so quality orriented as to even shred that 'rumoored defective" component. He had no reason to believe his original motherbaord was defective. Unfortun ately a majority never leanrn thow to thnk through a problem. Do not learn waht is defined in CSI: "Follow the evidence". Most randomly blame a part using speculation.
Most, without good diagnostic training, only shotgun. So we must retrain some techs to use a meter.