Internet Delay Measurements Using Test Traffic, Installing And Hosting A Test-Box

Abstract:

In RIPE-158, we presented a proposal to do Internet performance measurements using test traffic generated by test-boxes. This document is intended for ISP's interested in hosting a test-box. It describes the requirements for hosting a test-box at an ISP. It also describes the installation and day-to-day management of such a test-box. After reading this document, one should be able to judge if one can successfully host a test-box at a site. This document is not a formal request asking which sites are interested in hosting a test-box.

• 1.0 Introduction
• 2.0 Requirements for hosting a Test Box
  • 2.1 Rack space
  • 2.2 Antenna
  • 2.3 Network topology
  • 2.4 Local Contact
  • 2.5 Console
  • 2.6 DNS Issues
  • 2.7 Routing Issues
  • 2.8 Disclosure of data
  • 2.9 Form
• 3 First installation
  • 3.1 Installation
    • 3.1.1 Installing the hardware
    • 3.1.2 Booting the test-box
    • 3.1.3 The final steps
  • 3.2 Troubleshooting
• 4 Day to day management
  • 4.1 Contact information
  • 4.2 Security
  • 4.3 Reboots
  • 4.4 Moving test-boxes, power outages
  • 4.5 Disk problems
  • 4.6 Operating system upgrades
  • 4.7 Other hardware problems
  • 4.8 Insurance
• 5 Using the test-box as a Stratum-1 NTP server
• References
• About this document

1 Introduction

In a previous document [1], we described a proposal to do Internet performance measurements using test traffic. The test traffic used in this project is generated by test-boxes installed at the ISP's participating in this project.

This document lists the requirements that have to be met before one can host a test-box. It also gives details about installation and day-to-day operation of the test-box. This document will be updated as we gain more experience with the installation and operation of the test-boxes. The latest version will always be available from our website (http://www.ripe.net/test-traffic/) . This document is intended to solicit input from interested parties. It should provide all information that one might have to know before installing a test-box. If you think that our requirements are unreasonable or impractical, do not hesitate to contact us.

A formal request asking for sites interested in hosting a test-box has been sent around in the spring of 1998 and will be followed by a second request early 1999.

2.0 Requirements for hosting a Test Box

This section discusses the requirements for hosting a test-box. Please read this section and make sure that you can meet all requirements, before applying to host a test-box. An application sheet can be found online (http://www.ripe.net/test-traffic/Host_testbox) . If you cannot meet all requirements but still want to host a test-box, or want to discuss your specific situation before applying to host a test-box, do not hesitate to contact us.

2.1 Rack space

The RIPE-NCC test-box consists of a 19" wide crate. The crate is 4 u (approximately 20 cm) high.It is recommended to mount the crate in a standard 19" rack in a computer room or the like, though the crate can be located anywhere where one would consider installing a networked PC. However, one has to keep in mind that the test-boxes have to be connected to an external antenna, see section 2.2.

The operating system of the machine is FreeBSD. The security issues related to the installation of a test-box will be discussed in a separate document [2].

The test-box has to be connected to a 110 or 220 Vpower supply and the local network using 10base-T Ethernet. We will supply a power cable with a plug for a standard European, grounded, 220 Vsocket. In some places (the UK, for example) you may have to replace this plug with one that fits into your sockets.

2.2 Antenna

Figure 1: Placing the antenna on a building. The best choice is somewhere in the dark grey area, the second choice is somewhere in the light grey area. The white area is a third choice.

The test-boxes have to be connected to a GPS antenna. The antenna has to be mounted outside, with a clear view of of about half the sky (see figure 1, 1600,% elevation 500).% If there are local obstructions, try to find a location such that the antenna has a clear view to the south.

Our experience with placing antennas so-far shows that an antenna sitting on the inside of a window will often work but that putting the antenna just outside the window is generally a major improvement on the reception conditions. The antenna will not work inside buildings made of reinforced concrete or near window-frames made of metal. High buildings in the neighborhood of the antenna may have a negative impact on the reception conditions. Finally, our experience shows that the reception conditions often improve by moving the antenna around by a couple of meters. Therefore, if you have to pull cables in advance, please make them a bit longer than absolutely necessary.

We only support the direct mount option (see figure 2), as support for the other mounting options has been discontinued by the manufacturer (Motorola). The post-mounting bracket, though no longer available, can serve as an example in case you want to mount the antenna on a pole. The antenna has to be mounted (roughly) horizontally.

The antenna comes with about 6 mof (RG-58) antenna cable. This cable can be extended by a few meters using RG-58 cable. Longer antenna cables are possible but in this case one has to use low-loss coaxial cables such as Aircell7. So-far, we have been able to successfully run test-boxes with about 40 m of cable and otherwise good receiption conditions.

Please estimate the necessary cable length and fill out the number on the application sheet, so that we can provide the correct cables with the test-box. However, for calibration purposes, the length of the antenna cable has to be known. Therefore, before installing any antenna cables, measure their length.

In order to optimize our installation schedule, please indicate the time needed to install the antenna.

If your site has any special requirements for the installation of the antenna, for example lightning protection or other safety requirements, please indicate this on the application form. This field can also be used for questions regarding the antenna installation.

2.3 Network topology

As discussed in a previous document [1], the test-boxes should connected hops away from a border router, or, if that is not feasible, as close to the border router as possible. On the information sheet, please provide a short description or drawing of your local network situation.

In the same document, we also discussed the case where an ISP has more border routers. If this applies to you, please provide a list of possible locations. We will then discuss the preferred location of the test-box with you, taking into account the location and connectivity to test-boxes at other ISP's.

2.4 Local Contact

Figure 2: Mounting options for the GPS antenna. We will support the magnetic mount and direct-mount options (first and third option from the left).

Although we plan to control the test-boxes from a central point with no operators or service required at the local sites, we do expect that each site that hosts a test-box appoints a local contact. This contact should take care of things that cannot be done remotely, such as rebooting the machine or copy information from the console in case of network or hardware problems. The local contact, obviously, has to be reachable by phone or email.

In an ideal situation, the contact would be reachable 24 hours a day, 7 days a week (``24/7'') and could take care of our email's and phone-calls immediately. This could be the case at, for example, a place with operators around on a 24/7 basis. However, we do not think that there are many places where this ideal situation will exist.

The minimum requirement for the local contact would be that he is reachable during office hours (9am-5pm, Monday to Friday, except for public holidays), is able to respond to our requests within a day and can appoint a replacement during holidays and the like.

In the latter case, if a test-box causes trouble outside the hours where the local contact is reachable, it will removed from the measurement network until the problems can be solved.

There several possible intermediate steps between the minimum requirements and the ideal situation possible. In general, the shorter the time between a request from our side and an action from the local contact, will increase fraction of time that the measurement network is active (``duty-cycle''). The higher the duty-cycle, the more that can be derived from the data.

The local contact persons will be automatically subscribed to the mailinglist [email protected] . This mailing list will be used for communication relevant for all test-box hosts. The email address of the local contact will also be used for communication regarding the results of the measurements done with the test-box. If you prefer that this email is sent to another address, please let us know.

2.5 Console

The test-boxes do not have and, for normal operation, do not need a console. They will be delivered with the correct IP and Gateway numbers pre-configured and, when powered up, automatically connect to the network assuming that real configuration is the same as the one described on the information sheet.

However, for trouble-shooting it is useful if you can provide a standard PC monitor and keyboard that can be connected to the test-box. That way, it might be possible to solve problems while the test-box is at your site. If no console is available, then the only solution will be to send the test-box back for repair.

2.6 DNS Issues

The hosting organization should provide an IP number and name for this box, following your normal numbering scheme. This name has to be entered in the DNS by the host as an "A"-record. The name will be aliased to a name of the form:

ttXY.ripe.net

where XY is a number starting at . This will be set up by us.

The host should set up the in-addr mapping shortly after the domain name is known. In order to configure the test-boxes, the host should also provide the IP address of at least one topologically nearby resolving name-server.

2.7 Routing Issues

The test-boxes are set up to listen to the RIP and IRDP protocols. If those protocols are not available on the local net, then the address of a static gateway should be provided.

2.8 Disclosure of data

In section 4.1 of [1], the disclosure of data was discussed. We realize that this is a delicate matter and that all sites hosting a test-box should agree on the policy in order to make the project a success. We therefor have a separate [4] document discussing the data-disclosure policy.

How one can obtain a copy of the data as well as format of the data will be published on our web-site (http://www.ripe.net/test-traffic/) .

2.9 Form

If you meet all the requirements, please let us know by filling out the application sheet. This sheet can be found online at: http://www.ripe.net/test-traffic/Host_testbox. Again, if you cannot meet all requirements or want to discuss your specific situation first, please do not hesitate to contact us.

3.0 First Installation

This section describes the first installation of a test-box. Figure 3 shows a drawing of a test-box that should help you to locate the various connectors on the test-box.

This section will, as we get more experience with the installation of our test-boxes, be updated. Before you begin with the installation, please visit our web-site (http://www.ripe.net/test-traffic/) and download the latest version of this document. Site-specific details will be discussed by email.

3.1 Installation

3.1.1 Installing the hardware

When the test-box arrives check it for obvious shipment damage before mounting it into a rack. If the test-box has been damaged, notify the delivery company as well as the RIPE-NCC. Depending on the nature of the problem, we will then decide how to proceed. Do not switch on a damaged test-box.

On the back of the test-box, you will find a grey flat-cable with a DB9-connector. This cable has to be connected to the left DB9-connector on the back of the machine. If the cable has been disconnected during shipping, please put it back in.

Mounting the antenna is discussed in the previous section. Depending on your particular situation, antenna cables and connectors may be included with the test-box. If necessary, one can open the antenna and unplug the cable by removing the black, plastic, nut on the bottom of the antenna.

If your site uses Aircell7 cable, please note that, although the cable is quite heavy, it is also quite fragile. When installing it, please avoid making sharp bends, do not squeeze the cable and do not pull it through a large number of holes at once.

For calibration purposes, we have to know the length of the antenna cable with an accuracy of about1 m,so if you use cables other than the ones provided by use, please measure their length before installing them. It might be necessary to move the antenna a bit in order to optimize reception, so do not cut any cables or fix the antenna in a permanent position yet. The antenna cable has to be connected to the BNC plug on the back of the crate (figure 3).

Then connect the test-box to Ethernet using the RJ-45 connector on the back. If your site uses BNC connectors for the local Ethernet, get an adapter. Do not use the BNC connector on the back to connect the box to the local Ethernet, this may seriously damage your routers as well as the test-box.

The test-boxes use 2 different brands of power-supplies. The first one will recognize the voltage automatically, the second one (recognizable by the two green LED's just below the main-connector, has to be switched to the right voltage. The default is 220 V.If your site use110 Vset the TWO red switches below the LED's and chrome handle to 110. After that, connect the test-box to a power outlet.

3.1.2 Booting the test-box

Now switch on the test-box. A console monitor can optionally be attached to the test-box to monitor the boot process but this is not necessary. The test-box will boot automatically.

There is no need for the local contact to configure the test-box, as the test-box will be pre-configured at the RIPE-NCC before it is sent to you. Also, system maintenance and further configuration will be done from the RIPE-NCC.

When the test-box is running (approximately 3 minutes after it has been switched on, or when a login prompt appears on the screen), check if it is visible from your local network by pinging it at the pre-configured address. If it doesn't respond, first check the green status LED on the Ethernet card. If it is off, check your cables, reboot the machine and ping it again. If the LED is on, attach a console monitor, reboot the machine and write down any errors that appear. After that, please contact us and we will try to solve the problem.

As soon as the box is on the net, please send an email to [email protected] . We will then check the performance of the box and the antenna position remotely, and, if necessary, contact you in order to fine-tune the installation. As soon as the box starts taking data, plots with the results from the last days will appear on our web-site. We will email the local contact of the box with the URL as well as the necessary passwords. If other people inside your organization should have access to the results, please let us know.

3.1.3 The final steps

When the optimal antenna position has been found, we suggest that you fix the antenna at this position. The easiest way to do this is with cellotape. Alternatively, you can unscrew the base of the antenna, drill a hole and then put the antenna back together. If you have to unplug the antenna cables for this, please inform us in advance.

The front panel of the test-box can be locked. After you have powered up the test-box, we suggest that you lock the front panel, remove the key from the lock and put the key in a safe place away from the test-box. This will make it harder, though by no means impossible, to tamper with the test-box.

3.2 Troubleshooting

We will, as we get more experience with the installation of our test-boxes, put a list with questions and problems, as well as their solution, related to the installation of test-boxes on our web page.

4.0 Day to day management

4.1 Contact information

All email regarding the day-to-day operations of test-box should be sent to:

[email protected]

This is a mail-alias for the entire test-traffic group at the RIPE-NCC. Although we will read and respond to email sent to our personal mail-boxes, the response time to email sent to [email protected] will, in general, be much shorter.

We prefer email for all communication regarding the test-boxes. In exceptional cases, we can, however, provide you with a contact phone number or ``24/7'' contact address.

4.2 Security

There are several security related issues involved in day to day management. These are discussed elsewhere [2]. We assume that the local contact has read and will follow (the spirit of) this document.

4.3 Reboots

First of all, do not reboot the test-box unless you are asked to do so by the RIPE-NCC. The test-box can be rebooted by pressing the reset button. The reboot process will take a couple of minutes and can optionally be monitored by attaching a console monitor. The test-box is running again when the login prompt appears or when the box can be ping-ed from another machine.

4.4 Moving test-boxes, power outages

If a test-box has to have its IP or gateway numbers changed, please provide us with the new numbers and the day that they become effective as soon as possible but at least a week in advance. We will then reconfigure the test-box remotely, shut it down and let you know when it can be moved. After the test-box has been moved, follow the procedures for first installation.

If a power outage is planned at your site, please notify us a couple of days in advance. We will then do an orderly shutdown of the test-box shortly before the power outage. After the power is switched back on, the test-box can be booted as usual.

4.5 Disk problems

Our test-boxes are equipped with two swappable hard-disks. The data from the top (master) disk is copied to the second (slave) disk at regular intervals. If the disks are swapped

In case of a disk related problem, we will ask the local contact to swap the disks. First switch off the machine. Use the second key to unlock the disks and carefully pull them out of their slots using the black handle. Insert the disks in the other slots and lock them again. Then switch on the machine again.

We will first try to reformat the second disk and start using it as a slave. If this does not work, we will ask the local contact to switch off the machine, unplug the second disk and send the entire unit back to the RIPE-NCC.

4.6 Operating system upgrades

If we decide to upgrade the operating system, we will download and install the new kernel on the second hard-disk. After that, we will ask the local contact for a suitable time to swap the disks. At this time, the local contact has to switch off the machine, swap the disks as described in section 4.5, reboot the machine and optionally monitor the boot process. As soon as the machine is up again, we will check if the new kernel works as it should. If not, then the local contact will be asked to swap the disks back and reboot the machine again.

4.7 Other hardware problems

In the first instance, test-boxes with hardware problems not related to the disks have to be dismounted and sent back to the RIPE-NCC for repair. Unless the problem might be related to the GPS receiver, the antenna does not have to be dismounted.

As we get more experience with our test-boxes, we might provide instructions to the local contacts for doing small repairs.

4.8 Insurance

Although the machine remains our property, it is extremely expensive and often impossible for us to insure it against theft or fire. On the other hand, most sites have a general insurance for their equipment. If you have such an insurance, please include this machine for a value of 2000 Euro.

5 Using the test-box as a Stratum-1 NTP server

Our test-boxes run the NTP [3] software and can be used as a stratum-1 NTP server for your local machines. If you want to do this, add the following line to your NTP configuration file (usually /etc/ntp.conf):

server hostname prefer

and restart the NTP deamon on that machine.

6. References

1. 1 H. Uijterwaal, O. Kolkman, ``Internet Delay Measurements using Test-Traffic, Design Note'', RIPE-158.
2. 2 O. Kolkman, ``Internet Delay Measurements using Test-Traffic, Security Document'', RIPE-179.
3. 3 NTP version 3-5.92 or later, the software can be downloaded from http://www.eecis.udel.edu/ ntp/
4. 4 H. Uijterwaal, ``Internet Delay Measurements using Test Traffic, Data Disclosure Policy'', RIPE-180.