How to Properly Benchmark Your PC

Posted 01/27/2010 at 9:00am | by Gordon Mah Ung & Loyd Case

12

Comments

Print

Benchmarking the GPU

Benchmarking graphics cards gets more difficult every year. In the old days, most people had 4:3 CRT displays, and usually ran games at relatively low resolutions. Today’s gaming environment is considerably more complex. We’ve now got wide screen displays running in a variety of resolutions and two different aspect ratios (16:10 and 16:9).

Then there’s the confusing issue of APIs. It’s true that OpenGL for PC gaming is less relevant than it used to be, but now we have multiple versions of DirectX: DirectX 9, 10 and 11. Each offers different capabilities and feature sets. Currently, only AMD offers DX11 class GPUs, though that may change by the time you read this.

The next layer of complexity is figuring out which benchmarks are relevant. FutureMark’s 3DMark series has been popular for doing quick tests, but it’s also understood that a 3DMark score doesn’t always reflect how a particular GPU might perform in real games.

Then again, game benchmarks don’t always reflect reality, either. Take, for example, Far Cry 2. This Ubisoft title has one of the best built-in benchmarking tools we’ve seen. The problem is that it’s actually multiple benchmarks.

Far Cry 2 offers rich benchmarking opportunities – almost too rich.

So which benchmark is more useful? The longer, “Ranch Long”, which is almost purely a graphics test? The “Playback (Action Scene)”, in which AI and physics has a major role? Or the “Ranch Small”, which is more balanced between CPU heavy and GPU heavy elements?

If what you really want to do is test purely graphics performance, you’d not only choose the “Ranch Long”, you’d also take pains to disable AI and other CPU elements in the Game Settings tab. If what you want is to check out performance during actual gameplay, the Action Scene might be better – but CPU performance would have a large effect. On top of all this is the fact that a particular graphics card performance in Far Cry 2 might not reflect how it performs in a completely different game.

Then there’s the resolution question. Again, if you’re looking to just hammer on the GPU, run at very high resolutions. Then, add to the GPU’s pain by pumping up anti-aliasing and maxing out game detail and effects. So the right answer would be to use a big display – say, a 30-inch monster running at 2560x1600, right?

Well, not necessarily – not everyone has a 30-inch, 2560x1600 display. For example, if you take a look at the Steam user survey, you’ll see that the largest single group of users – over 20% -- is running at 1280x1024. That’s a 5:4 aspect ratio, and a pretty undemanding resolution. In addition, a $100 graphics card will completely tank at very high resolutions, while delivering very playable frame rates at something like 1280x1024.

The right answer, of course, is to run multiple different benchmarks at multiple resolutions. But you do have take a stand and minimize the number of variables, or you’d spend all your time benchmarking games and not actually playing games. If you’re testing your own personal setup, you’ll of course be limited by your CPU and your display. In our labs, we take a middle ground. We do test at multiple resolutions and using many games, but we also try to find the sweet spot for each card. When we benchmark a dual GPU card like the Radeon HD 5970, we’ll run it on a 30-inch display. On the other hand, we might max out at 1680x1050 for a sub-$100 card, but also test at lower resolutions, like 1280x720.

We’ve already discussed how to set up your system for more reliable benchmarking. What works for CPUs and systems also works for graphics cards, with one additional wrinkle: make sure you’re running the latest graphics driver. Performance in some titles can go up by over 10% just with a driver update (though in rare cases, you may actually see performance decreases.)

The good news is that you can find a pretty good set of graphics benchmarks that cost you nothing but download time and some time to learn. If you want to spend $50 for a particular game to run as a benchmark, feel free. But here’s our handy guide to a few of the good, free graphics tests.

S.T.A.L.K.E.R.: Call of Pripyat

The developers at GSC Game World have released two games based on their excellent S.T.A.L.K.E.R. series. The latest is the Call of Prypiyat test, which supports DirectX 9, Direct 10 and DirectX 11.

Call of Pripyat supports the latest APIs and is fairly simple to run.

Call of Pripyat is a first person shooter, but there’s no actual action in the benchmark, though AI is active. You’ll see characters walking around, but not actually engaged in combat. Since it tests multiple APIs, it’s good for finding the sweet spot for your particular GPU. CoP, as it’s often called, offers a fairly simple set of parameters: pick an API, pick a detail level and, if you like, enable AA and other features.

DiRT2

DiRT2 is the latest in the series of racing titles that started with the original Colin McCrae game. The free, downloadable demo. DiRT2 is a recent game that support DirectX 11, 10 and 9, so is about as current as you can get.

The problem with DiRT2 is that it’s somewhat cumbersome to run as a benchmark. First, you need to run the demo. Then you have to navigate the menu to find the options screen. You start inside your in-game RV trailer, have to leave the trailer to go outside, find the options table, and then select graphics. After all that, you still need to scroll down to the bottom of the graphics options to find the benchmark mode.