Our test suite for gateways and routers consists of an in-depth, network-analysis test using NetIQ's Chariot 4.3, a hand-timed file-transfer test, and an interior-range test.
Performance tests
Environment
We test throughput and response time for gateways and routers in a multinode network environment. We employ three Dell Dimension desktops for Ethernet trials and one Dell Inspiron notebook for wireless trials, all running Windows XP. For Ethernet tests, two desktop clients comprise an end-point pair. A third desktop runs the Chariot console, with which we execute and monitor tests between end-point pairs. For wireless tests, an end-point pair consists of the notebook client and one of the desktop systems. If the gateway/router has an Ethernet switch, then a third desktop system acts as the Chariot console. Otherwise, we run the console on the desktop system of the end-point pair. For information on setting up your own end-point pair and running your own test, see our guide to testing your own network.
Configuration
For our file-transfer and Chariot tests, we place the test systems within 10 feet of each other. They have an unobstructed line of sight, and the area is free of RF interference in the device's operating spectrum. For example, 802.11b, HomeRF, and Bluetooth adapters all operate in the 2.4GHz spectrum and are subject to interference not only from each other but also from devices such as microwave ovens and cordless phones. Thus, we conduct our tests in an area void of such devices. Under these conditions, we achieve maximum signal strength, so our results reflect a best-possible-case scenario. As signal strength fades, many devices negotiate with the clients to transmit at slower speeds. Throughput speeds will decrease as range increases.
Tests using NetIQ's Chariot 4.3 software
CNET Labs uses NetIQ's Chariot 4.3 and Performance Endpoints 4.4 to test throughput, transaction rate, response time, and lost data. We use Chariot's standard throughput and response-time scripts for general testing and run the tests using various protocols. The scores you see in our reviews correspond to results from tests that were run using TCP and UDP.
In addition, we analyze exceptionally poor or anomalous performance of any gateway/router by using scripts included in the Chariot test suite. This allows us to provide detailed information about the source of a particular device's subpar performance.
We use Chariot's streaming scripts to analyze devices with enhanced quality-of-service (QoS) capabilities. In such cases, the technology determines the type of supplementary testing we perform; occasionally, slight modifications to a test script are required to reveal a product's peak performance capabilities. For example, devices with enhanced QoS, such as those based on ShareWave's Whitecap technology, demonstrate better performance with send- and receive-buffer sizes of 32,120K. Any test-script alterations we make are noted in the performance charts accompanying the review.
In addition, we analyze exceptionally poor or anomalous performance of any gateway/router by using scripts included in the Chariot test suite. This allows us to provide detailed information about the source of a particular device's subpar performance.
We use Chariot's streaming scripts to analyze devices with enhanced quality-of-service (QoS) capabilities. In such cases, the technology determines the type of supplementary testing we perform; occasionally, slight modifications to a test script are required to reveal a product's peak performance capabilities. For example, devices with enhanced QoS, such as those based on ShareWave's Whitecap technology, demonstrate better performance with send- and receive-buffer sizes of 32,120K. Any test-script alterations we make are noted in the performance charts accompanying the review.
File-transfer tests
We hand-time the transfer of a directory containing 100MB of varying file types via the gateway or the router over an Ethernet connection. Results for hand-timed transfers that you run at home should closely match ours if you are testing at the fastest connection speed and the maximum signal strength.
Range tests
Environment
CNET Labs performs its range tests in our editorial and production offices, a real-world office environment. Our interior tests do not represent range capabilities across all environments. You may get better results in an open-air environment, a spacious facility without walls, or outdoors. An area such as ours, divided by many walls, most likely leads to substantially decreased range. Our tests, therefore, provide a benchmark for estimating the range of WLAN devices in indoor settings with a relatively limited number of RF obstacles.
Configuration
We set the wireless adapter's transmit power to its highest setting and its connection speed to its maximum. For a typical 802.11b card, these settings are generally 30mW transmit power and fixed 11Mbps throughput, although some cards may offer greater transmit power. Adapters based on other technologies or standards, such as 802.11a, may be capable of faster connection speeds.
Measurement
To calculate a product's communication range, we measure the greatest distance in 10-foot intervals (rounding up at 5 feet) at which the device can complete a Chariot filesndl.scr script between the gateway/router and a single connecting notebook at maximum connection-speed settings.
