Re: New Document available: RIPE-271 (fwd)

[Roberto Percacci <]

On Thu, 8 May 2003, Henk Uijterwaal (RIPE-NCC) wrote:

> A new document is available from the RIPE document store.
> 
> 
> 
> Ref:            ripe-271
> Title:          RIPE NCC Hostcount in the 21st Century
> Author:         Daniel Karrenberg
> Date:           8 May 2003
> Format:         PDF=98090 TXT=8008

Dear fellow TTM users,

following discussions at RIPE45 in Barcelona, I would like to offer some 
remarks on the "new business model" and attendant changes of policies 
concerning measurements and in particular TTM.

I generally agree with the analysis and the proposals made in RIPE-271.
However, I share other people's concern that the policy of total 
disclosure that is being proposed could slow down or even halt the growth 
of TTM.
In particular, I expect that ISP's will generally not be happy to see 
their performance data made public by third parties, even when the third 
party is trusted to be neutral and objective.
In this way, there is a risk of losing the commercial participants and 
retaining only the academic ones. 
If this happens, we may still have some good data for research, but the 
data will be clearly biased and less representative.
Furthermore, TTM may become largely irrelevant to the community of ISP's, 
which is the main constituency that RIPE is supposed to serve.

Also, I think that total disclosure of the data is neither necessary nor 
useful.
Suppose I am an ISP "A". In general there is very little I can learn about 
my connectivity from knowledge of delays involving ISPs B and C.
It seems to me that publishing a mountain of raw data would not
really be of much use to the general RIPE community.

On the other hand, if someone really needs all the data and gives 
appropriate guarantees of respecting privacy and anonimity, he can already 
have access to the data, as happened to me when I asked to perform some 
statistical analysis on them.

I believe that the current problem lies not so much in the lack of 
openness as in the fact that there is no useful summary of the data.
As stated in RIPE-271 "we have not developed enough...useful products for 
the RIPE NCC membership and the RIPE community at large".

My main suggestion is that RIPE publish on the web site only some 
statistical summaries of the TTM data. This would be of general usefulness 
insofar as it would give everybody a general feeling of what to expect 
from their Internet connection.
This would be a useful service, because I believe that Internet 
performance is still not so well understood. In particular, customers are 
often baffled by widely different pricing for apparently similar services.

Then, it should be clear that there is a unique "natural" disclosure policy:

1) everyone can see the summary of all the measurements, but only in 
statistical form (let's call this the "benchmark")
2) a TTM user can see the results of all the pings that start or end at 
her site, both in detail (useful for diagnostic purposes) and in 
statistical form (to compare with the benchmark). 
3) it is neither useful nor appropriate that anyone sees performance data 
between third parties (RTTs on paths that do not begin or end at one's 
site, or the statistics of other sites).

This policy seems to me to address the needs of all parties and should be 
universally acceptable.

To make things more well defined, many details need to be worked out.

One question is: what type of statistics to use? My suggestion is that the
ratio RTT/distance is the most useful performance metric, for various 
reasons:
- if RTT is expressed in milliseconds-light, this ratio is a dimensionless 
  quantity, as one would expect of a quality metric;
- it tells us how fast the packets travel compared to the physical 
 absolute bound given by the speed of light: it varies between one (best 
  possible performance) and infinity (no transmission)
- the distribution of this quantity appears to be dominated by a large 
  power-law tail, which makes it a mathematically interesting variable

I will not elaborate further on this. More details can be found in our
contribution at PAM2003.

Another way in which this could benefit the community at large is to give 
also non - TTM participants a way of gathering their own performance 
statistics, at the same time gathering much higher statistics.
This could work as follows: a user downloads a software that pings the TTM 
hosts for some time and then extracts quality statistics.
In addition, these data would also be sent to the central database and 
used in forming the "grand average".
I have spent some time thinking of this and I can provide more details
if necessary. The main point is that in some way a user (presumably an 
ISP, but perhaps in a later phase also an end user) would be able to 
derive quality statistics and compare her own quality statistics to the 
average benchmark published on the TTM web site.

How much would all this cost? If we had to install thousands of text boxes
all over the place, it could cost a lot. However, by storing all the data 
generated by other people that ping towards the TTM boxes, we could gather 
a lot of data at very low cost.
Eventually, the whole system could become a peer-to-peer network, where
every participant agrees to allow others to ping him in exchange for the 
right to ping them.
In this case the TTM boxes could become a small subset of the whole 
system, acting as some kind of initial core. RIPE would then simply oversee the 
system to ensure that the data are correct and, out of the data of all 
participating users, construct the benchmark.

I believe that in time this would become an extremely useful service 
and for reasons that we all know, RIPE is in the ideal position to set up 
and run such a system.

Best regards

Roberto Percacci