Well technically the processor is responsible for all video, it has to process the file and decode it to a form the video card can render. Oh and that takes memory too. I have yet to find a PC that plays video files without either of those components. Oh wait it also has to pull that video from some form of storage.
What I am getting at is any "weak link" in your PC could cause this issue not just the video card. Try using admin tools to check your processor and memory load. Also check your drives throughput. If that checks out then yes it's a video card issue.
Nope that banana already fell out of the bag.