How We Test Wireless Routers

avatar

How We Test Wireless Routers

Maximum PC Lab North is a 2,800 square-foot home located on 10 acres of what was once a dairy farm. We've only detected one other wireless network in the neighborhood, and its signal is exceedingly weak--it's certainly not strong enough to shut down channel bonding on the routers we test.

The exterior frame of the house consists of 2x6 Douglas Fir framing sheathed in fiber-cement siding. The foundation is concrete and the roof is clad in composite shingles. The interior walls are constructed from 2x4 Douglas Fir studs and are clad in drywall with R-19 insulation in the cavities. The interior ceilings are 10 feet high.

The Media Room is unusual in that it is a room-within-a-room. We went this direction to deliver superior acoustic performance with music and movie soundtracks. It consists of four walls of 2x4 studs clad in drywall inside four walls of 2x6 studs clad in two layers of drywall (with acoustic caulking sandwiched between). The ceiling also has two layers of drywall. There are two layers of R19 insulation stuffed into the walls and the attic is insulated with blow-in fiberglass material. Two solid-core Masonite doors (one opens out into the foyer and the other opens in to the room) further deaden the room and prevent sound from leaking. We added R-19 fiberglass insulation to the interior walls throughput the rest of the home the house contain R-19 insulation for acoustical reasons. You can see pictures of the room during construction here.

In order to be as fair as possible to each vendor's product, we retest our zero-point router on the same day we test each challenger's product. We can't guarantee atmospheric conditions will be exactly the same, but this gets us as close as we can. As a result, our zero-point numbers will change for each test. Iif we test several routers in one day, we'll obtain just one new baseline number. 

We use Trendnet's TEW-684UB dual-band USB adapter, because it's the only USB adapter we know of that supports three concurrent 150Mb/s spatial streams. In order to measure peak performance, we set the router to operate in 802.11n-only mode with WPA2 encryption and channel bonding enabled. We use the free utility Jperf (the Java front-end to Iperf) to measure TCP throughput to the client at four locations inside the home, one with the notebook in an enclosed outdoor patio, and one with the notebook completely outside the home. The JPerf server is connected to the router via a 24-port gigabit switch (both the router and the switch are located in a homerun).

For dual-band routers, we perform the same battery of tests on both radios. And for routers that support attached storage devices, we use the same methodology we use to benchmark NAS boxes and home servers: We use a stopwatch to measure how long it takes to write first a single 2.79GB file to the storage device and then a large collection of small (files from the CD distributed with the November 2007 issue of Maximum PC). We repeat this test to read the large and small files from the storage device. We use a desktop PC hardwired to our homerun in both instances.

Here are the locations where the router and the notebook client are placed during benchmarking.

ROUTER LOCATION

The router is placed on top of a wooden chest of drawers, 50 inches above the floor.

LOCATION 1: BEDROOM

The notebook PC and client adapter are on a wooden table nine feet from the router. 


LOCATION 2: KITCHEN

The notebook is located on a granite countertop (45 inches above the floor and about 20 feet from the router), with one insulated wall and a set of plywood cabinets separating the PC from the router.

LOCATION 3: MEDIA ROOM

The notebook is on a coffee table inside the media room, about 35 feet from the router. There are five walls (three of which are insulated) separating the notebook from the router. (The doors are an exterior type in order to reduce sound leakage.)

LOCATION 4: ENCLOSED PATIO

The notebook is located on a glass table inside an enclosed patio (30 inches above the floor and 38 feet from the router). The patio is not part of the conditioned space of the home, so the signal must pass through one insulated exterior wall in addition to one insulated interior wall and a set of plywood cabinets.

LOCATION 5: OUTDOOR LOCATION

We've done a little landscaping since we started testing wireless routers here. We drove a piece of rebar into the ground to mark where the client should be placed for benchmarking.

OUTDOOR VIEW OF THE ROUTER'S LOCATION

NEAREST NEIGHBOR

The closest house is quite a distance away, so there shouldn't be any router conflicts that would cause problems with channel bonding. (That's part of a grid-tied solar system in the upper right corner.)

 

Editor's Note: We've made a number of changes to our testing procedure since we first started testing wireless routers at this location. Here's a recap:

Effective August 31, 2010, we added a bedroom location closer to the router.

Effective March, 2010, we're using JPerf Vers. 2.0.2.

Effective March 2010, we've switched from PC Card Wi-Fi client adapters to USB Wi-Fi client adapters.

Effective April 15: 2008, we connected to server running JPerf to a switch via the homerun, as opposed to wiring it directly to the router's switch. The resulting impact on TCP throughput has proven to be negligible. We also eliminated the Home Office location from our test sequence.

Note: Effective January 15, 2008, we added two new test locations within the home: An enclosed patio and a bedroom.

7

Comments

+ Add a Comment
avatar

LeonThePC

In the October 2011 issue of Maximum PC, the Doctor also talks about using Jperf to measure network performance. But it is not mentioned that it is only offered as source code.

Can you give a link to a precompiled version of Jperf and iPerf or can you give step-by-step instructions on how to compile them?

avatar

MrMick

You can download the binaries from Sourceforge. Here's a link: http://iperf.sourceforge.net/

avatar

kingsley

Hi

avatar

ICEinLAVA

So now that you created the test environments, where's the results? or is about 4 months not enough time to take accurate benchmarks?

avatar

MrMick

The purpose of this article is to explain the methodology we use for testing wireless routers. Each time we publish a review, such as the Linksys WRT600N , we'll link to this story so the reader can gain an understanding of how we got our numbers.

Here's a link to the WRT600N review

avatar

MrMick

Broadband Internet access was my biggest concern when I moved into my new house, because Comcast (the "in-town" cable provider) didn't offer service outside the city limits, and AT&T didn't provide DSL.

I was about to subscribe to a Wi-Max service (ClearWire had just started operating in the area), but I thought I'd check with AT&T one more time. As it turned out, the company had decided--in response to ClearWire, no doubt--to add a few more central offices outside the city limits. One of these was less than 100 feet from my driveway (although my house is set back about 300 feet from the street).

In any event, I now have 6Mb/sec DSL service. It's not nearly as fast as Cable, but I'm satisfied.

Michael Brown
Executive Editor

avatar

harbingercmt

Just read your router article in the January edition of MPC. What caught my attention was your new house being built on what was once a dairy farm. I grew up in a rural area till much of it was paved over with new houses, strip malls, etc. Even though the crush of development and the problems it brought (transportation, taxes, crime, etc.) were a real bummer; I had hoped we would finally get an alternative to dial up internet. There is a satellite solution, but from what I've seen - it is only marginally faster than dial up.

So here's my question: How does an employee for MPC put up with painfully slow internet? Is there a sweet DSL, cable, or (dare I say) fiber optic option that you have somehow been able to acquire?

Thanks much in advance

Log in to MaximumPC directly or log in using Facebook

Forgot your username or password?
Click here for help.

Login with Facebook
Log in using Facebook to share comments and articles easily with your Facebook feed.