Providing Test Traffic Measurements (TTM) As A Membership Service

RIPE-209

Publication date:: 23 Oct 2000

State:: Obsoleted

Author

Henk Uijterwaal

File(s)

PDF (192.5 KB)

Abstract:

This document describes the model for selling Test Traffic Measurements (TTM) as a membership service. This document is based on discussions at the various RIPE-meetings as well as input from the RIPE NCC lawyers.

The actual sale of test-boxes as well as the services provided by the RIPE NCC will be subject to the ``General Terms to the RIPE NCC Test Traffic Measurements Service and Test-Box Purchase Agreement''. These will be published as a separate document. In case of any differences between this document and the ``General Terms to the RIPE NCC Test Traffic Measurements Service and Test-Box Purchase Agreement'', the latter document will take preference.

1 Introduction

Since early 1997, the RIPE NCC has been working on the Test-Traffic Measurements project (TTM). The goal of this project is to do independent measurements of Inter-Provider network parameters, such as One-way-Delay, Packet-Loss or Routing vectors, as specified in RFC's 2330, 2679 and 2680 [1]. The design goals have been described in an earlier document [2]. Several papers about the implementation and first results of the project have meanwhile been published [3].

The RIPE NCC has always used a model where new, experimental projects were funded from the general budget. However, once a project becomes more mature and can be offered as a regular service to its membership, the costs for the service are charged to the sites using the service.

So-far, the TTM project was funded from the general budget, as, when the project started, it was not clear if the project proposed in the design document [2] could actually be implemented. This situation has changed by now, the RIPE NCC has proven over the last years that it can routinely do delay- and loss-measurements on a large scale.

A direct consequence of the funding model was that participation in the project was limited to current RIPE NCC association members. After all, they funded the project. Also, as the number of ``test-boxes'' (measurement stations) was limited, we only installed 1 test-box at each member in order to give as many sites as possible a chance to participate.

Over the last years, we have seen that these limitations are too restrictive. First of all, a fair number of sites which are currently not RIPE NCC members are interested in hosting one or more test-boxes. These sites usually do not need the LIR services offered by the RIPE NCC, because they either are not LIR's or already receive these services from another RIR.

Then, several large customers with more than 1 PoP have expressed interest in hosting more than one test-box in order to include their internal networks in the measurement chain as well, or to provide their customers with measurement data between each PoP and the outside world.

For all these reasons, the RIPE NCC, in consultation with its AGM [5], has decided to move the TTM from an experimental project to a service that will be offered to the entire community.

This document describes the model for offering TTM as a service. In the first half (section 2) the principles behind the operation are described; the second half (section 3 onwards) describes the implementation details as well as the service agreement between the RIPE NCC and the site participating in the project.

1.1 Service Contract

Sale of the test-boxes as well as the service offered by the RIPE NCC, will be subject to the ``General Terms to the RIPE NCC Test Traffic Measurements Service and Test-Box Purchase Agreement'' [4]. In the following sections, we will summarize that document, as well as give an extensive overview of what is being offered as part of the TTM service.

This document was accurate and complete when it was published. However, the ``General Terms to the RIPE NCC Test Traffic Measurements Service and Test-Box Purchase Agreement'' [4] may have been changed since then and what is currently included in the TTM service may have changed as well. Before you buy a test-box, you will receive a copy of [4] for you to review and sign.

In case of differences between this document and the copy of [4] signed by you, the latter document will take preference.

2 The basic model

The basic model for participating in the TTM can be described in 2 paragraphs:

First of all, the site interested in participating in the service orders one or more test-boxes from the RIPE NCC. These test-boxes will be installed at the site of this organization. The host site pays for the hardware of the test-boxes and the test-boxes become the property of the host site.
In parallel, the organization pays a yearly service fee for the operation of the test-boxes. This service fee covers the cost of operation of the test-box by the RIPE NCC, analysis of the data, software updates, access to all products based on the TTM-data and everything else needed to keep the box running. Annually, the host site can decide to renew the service contract.

Sale of the test-boxes as well as the service will be subject to the ``General Terms to the RIPE NCC Test Traffic Measurements Service and Test-Box Purchase Agreement'' [4].

2.1 Cost of a test-box

A test-box currently costs approximately Euro2,500.- *, installation costs another Euro250.- for a total of Euro2,750.-. The installation and shipping costs will be reduced if a site orders more than one test-box in a single order. Shipping, local taxes, import duties, and such will have to be paid by the host site.

The test-boxes have a 1 year warranty. The RIPE NCC will buy a few spare test-boxes. In case of a hardware failure, the box will either be repaired or exchanged with a similar test-box from the spares, the latter only if this is possible without having to re-import the test-box.

More details can be found in section 3.

2.2 Service fee

No service fee will be charged in the year 2000. The AGM [5] has asked that the TTM service should generate a self-sufficient budget in 2001. Our first estimates show that this implies that the service should generate approximately Euro300,000 in service fees in that year. Assuming that we increase the measurement network to about 100 nodes in 2000, this implies a service fee per test-box of Euro3,000. Volume discounts for sites hosting several test-boxes will be discussed at a later stage.

The service fees are intended to recover those costs which the RIPE NCC incurs by providing the TTM service. These are still somewhat hard to predict exactly and adjustments will be made in the coming years according to experience. The expectation is that the service fees will drop as a consequence of more participants sharing the total cost.

At the moment, it is not foreseen that the service fee will include membership of the RIPE NCC association. However, recent developments have indicated that offering a service without membership may be in conflict with the by-laws or tax-status of the RIPE NCC. This is being investigated.

As the RIPE NCC and the RIPE NCC board do not want to lose momentum with the introduction of this service, it has been decided to set the service fee for 2001 to Euro0 for the time being. However, any new sites will have to sign the service contract to allow for the introduction of a service fee in the future.

The service fee covers operation of the test-box, access to the data according to the current data-disclosure policy [6] and products based on the TTM data as well as maintenance and support, as discussed in more detail in section 4.

2.3 Transition for existing test-box hosts

The RIPE NCC proposes that all sites, including the ones currently hosting a test-box, will pay the service fee when this is introduced. However, any hardware that has already been installed will be given to those sites for free. Existing sites will be asked to sign the service contract in order to create a legally binding agreement between the RIPE NCC and the site.

The RIPE NCC is aware that not all sites currently participating in the TTM, have budgeted resources for this service for 2001 and is prepared to discuss different transition plans for those sites.

There is no warranty on the currently installed hardware anymore. If anything breaks, the box can only be fixed against cost. The next generation of test-boxes will have clocks that are easier to operate and maintain. Once the RIPE NCC has some running experience with these clocks, it will offer the possibility to upgrade the existing (Motorola) clocks to the newer (Trimble) clocks against cost.

2.4 Private and public experiments

So-far, all measurements done with the test-boxes are what we call ``public experiments'', that is, the test-box is operated as a black box by the RIPE NCC, measurements are being done to other test-boxes hosted by other sites and the data is made available to both the hosts of the sending and receiving test-boxes.

However, there has been some interested in doing ``private'' experiments with the test-boxes, for example:

Measurements on the internal network, where the host site does not want to share the data with others,
Other experiments that can be done with the test-boxes, or
Measurements on experimental networks that are not connected to the global Internet.

Although the RIPE NCC wants to focus on public experiments, it does want to help sites to set up their private experiments. For this reason, test-boxes will also be sold without a service contract. The details of this agreement will be discussed with the potential customers at a later stage.

In order to guarantee the integrity of the data*, a test-box cannot be used for both public and private experiments at the same time.

In this paragraph, we have assumed that the number of requests for test-boxes for private experiments is small. If the number grows, we may have to revisit this model.

2.5 Build your own test-box?

Several sites have asked if it would be possible to build their own test-boxes instead of ordering them from the RIPE NCC. In some countries this might be cheaper, though one should keep in mind that the RIPE NCC orders a large number of test-boxes at the same time and thus gets volume discounts.

Setting up a test-box will also require more work (of the order of a day) and knowledge (in particular of the FreeBSD Operating System) at the hosting site.

The advantage is that setting up a test-box will sometimes be faster, as one does not have to wait for the RIPE NCC to produce a test-box.

If, in the future, it turns out that there is sufficient interest in this, the RIPE NCC will publish the requirements for a machine to be used as a test-box. Note that only a very small number of hardware configurations can be tested by the RIPE NCC. Any hardware not on the list will not be supported.

If the test-box will be used for ``public'' experiments, RIPE NCC will provide the host with instructions on how to configure the box. After the box has been set up, the host site should transfer control of the machine to the RIPE NCC. The host site should also sign a service agreement [4] for the test-box and pay the service fee. After these formalities have been taken care of, the RIPE NCC will then download the necessary software to the machine and start doing measurements.

2.6 Software license

The service contract gives the site hosting the test-box the right to use the TTM software on the test-box, to review the source code to detect any possible security risks that it may introduce at the hosting site and a few other basic rights according to Dutch law. The license to use the software expires when the service contract ends. Any other usage of the TTM software is not allowed.

However, the software can be licensed to be used for other purposes. For this case, we propose to use the GPL [7]. Details will be discussed in a future document.

2.7 Further development and feedback on the service

The TTM service will continue to be improved and expanded. The global development plans as well as the service fees for the next year will be described in the annual RIPE NCC Activity Plan. This activity plan is discussed and approved by the AGM of the RIPE NCC.

On a less formal level, the RIPE NCC will present its development plans for the TTM service in the RIPE Test Traffic-Working Group (TT-WG) meetings and mailing-list, where they can be discussed. These meetings and mailing list are open to everybody interested in this service.

3 Details on buying a test-box

Test-boxes can be ordered by filling out the online order-form with the necessary configuration data. After that, you will receive a copy of the service contract. This has to be signed by somebody authorized to do so in your organization. The RIPE NCC will also send an invoice with instructions on how to pay for the box.

We will assume that you have read the installation document [8] before ordering a test-box and that you have checked that your site meets the requirements necessary to install and operate a test-box. We also assume that installation of a test-box will not violate any local security or safety requirements.

After your order has been received, a test-box will be configured according to the current production schedule and tested. It will be shipped to your site as soon as the machine is ready and the payment has been received.

Included in the shipment are:

1 test-box according to the current specifications.
3 year warranty.
Pre-configured and tested at the RIPE NCC.
Installation support by email: the mailbox will be checked at regular intervals during working hours, with the initial response within 1 working day. Installation support includes: getting the test-box connected to the Internet, help with finding a suitable spot for the antenna and checking the configuration.

Not included with the test-box are:

A keyboard, monitor or mouse. These are not needed for normal operations but might be useful for troubleshooting. We assume that the host site can provide one for a short period during testing.
Taxes, import/export duties, etc. These will be charged to the recipient and may have to be paid before the test-box is delivered.
Any special or additional hardware (re-radiators, antenna splitters, cables, etc.) that might be necessary to operate the test-box under circumstances other than those described in [8]. The RIPE NCC has experience with test-box operations under special circumstances and can give advise here. However, any additional hardware needed in these cases will have to be paid by the host.
Setting up other hardware such as routers or fire-walls needed to operate the test-box.
Shipping to your site, see the next section for details.
Installation of cables and such.
On-site installation support by a RIPE NCC engineer.

3.1 Shipping

By default, the test-box will be shipped to the host site by SkyNet. The box will be insured for the replacement value of the test-box. The current rates for shipping the boxes can be found in table 1. These rates are an indication only and may change without notice, please contact the RIPE NCC for the latest rates.

As an alternative, the box can be sent to the host site by FedEx, UPS or another company with the costs directly charged to the customer's account. To use this option, please send us your account details.

Country		Rate
1.	Holland, Belgium	43 +	0.45x
2.	Other European Union Countries	130 +	4.50x
3.	Other European Countries	312 +	12.30x
4.	Middle East, Africa	406 +	15.90x
5.	Former Soviet Union	312 +	12.30x
6.	North America	219 +	8.20x
7.	Australia, Japan	292 +	11.80x
8.	Elsewhere	406 +	15.90x

Table 1: Costs of shipping a test-box to various destinations. The weight of a test-box with cables and such is between 20 and 25kg, the base rate is for a box of 20kg, x is the number of kilos above 20. These rates are the upper limit for shipping to a particular destination, in practice they might be lower. These rates do not include Dutch Sales Tax (of, at the moment, 17.5%). These rates are an indication only and may change without notice.

3.2 Site-surveys and installation by the RIPE NCC

The RIPE NCC has some possibilities to send an engineer to do either a site-survey or help with the installation of the test-box at a site. However, all travel and related costs as well as an hourly fee will be charged to the host site.

The RIPE NCC cannot do electrical safety tests and such. If those are required, please contact a local expert.

4 The service fee

The service fee includes operation of the test-box as a ``black box'' by the RIPE NCC. Under normal circumstances, the test-box host will not have to do anything to keep the test-box running. We do, however, expect that the site appoints a local contact person to help with problems that cannot be solved remotely. (For example, pressing the reboot button.)

According to [4], the RIPE NCC will operate the test-box for the hosting site, collect and analyze the data, and maintain the software, without giving any details on how this is implemented. This is done for practical reasons, if the services are specified in detail, any change would require that an updated contract is signed by both the hosting site and the RIPE NCC. This would severely limit the RIPE NCC's flexibility in responding to its customers' wishes.

At the moment, the service fee would include the following:

Delay/Loss measurements:
- To all sites considered interesting by the RIPE NCC. Hosts can suggest to include/exclude specific paths and/or change rates. However, the final decision will be made by the RIPE NCC.
- The latest version of the measurement software.
- Collection of data by the RIPE NCC once a day. All services and products will be based on data that could be collected by the central computer at the RIPE NCC between 2:00 and 4:00 GMT. If data could not be collected for a particular day, the RIPE NCC will try to collect the data from the test-box the next day until the files expire after 30 days. All data will be stored, but analysis will only be done retroactively for files not older than 7 days.
Services, currently (please see [9] for an overview of the current products and services):
- The raw data, in ROOT-format [10], will be put on our ftp-site on request. This includes a ROOT software library necessary to read the data.
- Daily, Weekly, Monthly plots of delays and losses.
- Delay alarms to a specified email address.
- Network Performance Scores when this service becomes available. (Planned for late 2000.)
- Overview plots for planning purposes when this service becomes available in 2000.
- All other products based on the TTM data.
Maintenance of the software.
- Software updates when they become available.
- Automated checks of the data collection processes every 3 hours and automated recovery/restarts as far as technically possible.
- Checks of log files once every working day, plus feedback from us to the test-box host in case of problems that require action from the host. (Data Quality Monitoring, everything else that involves human intelligence.)
Maintenance of the hardware.
- CPU: Trouble shooting by the RIPE NCC together with the local test-box contact person. The RIPE NCC will decide if a test-box has to be sent back for repair.
- Disks: will be replaced as long as they are under warranty.
- Clock: the warranty on the clocks will be transferred to the host site.
Email service to questions regarding test-box operations, with an initial response within 1 working day (24h).

Not included are:

An account on the machine.
Hardware or software changes are not allowed unless agreed beforehand with the RIPE NCC.
All repairs not covered by the warranty as well as shipping costs when a test-box has to be shipped back for repair. All costs will be charged to the host site.

Also, the service contract assumes that the host agrees with the data-disclosure policy. The service contract covers a period of 1 year, there is no obligation to use the service the next year.

The host site agrees that traffic from/to the test-box will not receive any special treatment in order to artificially improve the results of the TTM by that site. The RIPE NCC reserves the right to verify this or to stop providing the service if it believes that the site violates this agreement.

The host site also agrees to keep all contact information for the test-box up to date and inform the RIPE NCC in advance if a test-box has to be moved or the configuration has to be changed.

This list may be changed at any time, please contact us to see if there are any updates. The RIPE NCC is also always interested in expanding the TTM service, so any suggestions to extend measurements or analysis are more than welcome.

References

1: V. Paxson et al, ``Framework for IP Performance Metrics'', RFC2330, May 1998.
G. Almes et al, ``A One-way Packet Delay Metric for IPPM'', RFC2679, September 1999.
G. Almes et al, ``A One-way Packet Loss Metric for IPPM'', RFC2680, September 1999.
2: O. Kolkman, H. Uijterwaal, ``Internet Delay Measurements using Test Traffic'', ripe-158, May 1997, see: http://www.ripe.net/test-traffic/Notes/RIPE_158 .
3: Henk Uijterwaal et al, ``Internet Delay Measurements using Test Traffic: First Results'', in Proceedings of the SANE98 Conference, Maastricht, November 1998.
4: Jacqueline M. B. Seignette et al, ``General Terms to the RIPE NCC Test Traffic Measurements Service and Test-Box Purchase Agreement'', ripe-214, September 2000, see: http://www.ripe.net/ripe/docs/ripe-214.html .
5: ``RIPE NCC Annual General Meeting Minutes 1999 (AGM'99)'', ripe-205, see: http://www.ripe.net/ripe/docs/ripe-205.html .
6: Henk Uijterwaal, ``Internet Delay Measurements using Test Traffic, Data Disclosure Policy'', ripe-180, see: http://www.ripe.net/test-traffic .
7: GNU General Public License, see http://www.gnu.org/copyleft/gpl.html .
8: Henk Uijterwaal, ``Internet Delay Measurements using Test Traffic, Installing and Hosting a Test Box'', ripe-168, see: http://www.ripe.net/test-traffic/Notes/RIPE_168 . This document will be replaced shortly, check our website for the current document.
9: René Wilhelm, Minutes of the TT-WG Meeting at RIPE35, see http://www.ripe.net/ripe/meetings/ripe-35/ .
10: Rene Brun and Fons Rademakers, ``ROOT - An Object Oriented Data Analysis Framework'', Proceedings AIHENP'96 Workshop, Lausanne, September 1996, Nucl. Inst. & Meth. in Phys. Res. A 389 (1997) 81-86. See also http://root.cern.ch/ .

Footnotes

...Euro2,500.-: Prices as of October 1, 2000, please refer to http://www.ripe.net/test-traffic/Host_testbox for the current prices.
...data: For example, a private experiment with a lot of I/O, will affect the results of the regular measurements. This is hard to detect and correction afterwards is almost impossible.