Maximum PC

...and now I can't get it out. The only thing left I can think to try is to remove it from the case, hold it upside-down and try to shake it out. So my question is, do I really need to do that? Is it dangerous to just leave the screw in there?

Yes you should get the screw out, since there are conductive electrical components in there that could short out.

That's what i was afraid of. Thanks for the prompt reply.

I did the exact same thing when I was installing my GTX 560 TI. If your PSU is modular unplug all the cables at the PSU end and remove the PSU from the computer case. Get a bright flashlight or such to see if you can locate the screw. If you can fit a magnetic screwdriver between the grill covering the fan, try and get the screw that way. BE VERY CAREFUL, to not touch anything but the screw. I am a firm believer in not sticking magnets near electronics, but this is an exception.

I am suggesting this because, rattling around a screw trying to remove it, could damage stuff in the PSU. Good luck!

Out of eternal curiosity, what PSU is it? If its Corsair TX series, a small screwdriver could easily fit. (Personal Experience)