[mat-wg] some thoughts on add-path and bmp at ripe/ris

Let's see...

>>  o once a collector commits to ADD-PATH, it's a promise to keep
>>    the data forever; well, it's been 26 years of RV and a few
>>    less for RIS.
>>    - Did operators find the ADD-PATH data sufficiently useful
>>      to be worth the cost?  How did they use the ADD-PATH data
>>      and how often?
>>    - Did researchers find the ADD-PATH data sufficiently useful
>>      to be worth the cost?  How did they use the ADD-PATH data
>>      and how often?
>>    - Did folk develop special tools to take advantage of
>>      ADD-PATH data?

This is hard to answer... I know for sure that a couple of research groups used Isolario data for their studies, in particular researchers at Max Planck Institute where studying ADD-PATH data we collected, but I'm not sure on the final outcome of their research. Also, data was used by Hurricane Electric for their bgp.he.net tool. Besides that, I am not aware of operators that used the data, but I never focused my attention on access logs back in the days.

>> o On the problem of stream/peer identification.  You describe
>>     the divergence of BIRD and FRR.  What about commercial
>>     router vendors?

This may be totally wrong, but I remember that Cisco was not allowing ADDPATH for eBGP. Not sure about other vendors though.

>>  o You say data were often redundant, though not fully.  Did
>>    you investigate mechanisms to reduce the storage, or did you
>>    see that as a path to complexity and fragility merely to
>>    save some spinning rust?

We did investigate that. Memory-wise, we studied methodologies to exploit compressing technique such as LZW (see Interactive Collector Engine (ripe.net)<https://ripe75.ripe.net/presentations/21-pres.pdf>). However, the routing engine we devised for route collecting + the amount of RAM we had in the collectors were enough and never required us to turn that feature on. Disk-wise, we found out that xz was a great candidate to replace bz2 and gz as the next compressing methodology for route collecting. I don't remember the results to be honest, but that should be easy to reproduce. However, we decided to not compress data in xz because only bgpscanner was able to read xz data - hence anyone that wanted to use other tools such as bgpdump would not have been able to read them.

>>  o You mention the withdraw storm between your peer and their
>>    peer (and later an announcment storm, I presume).  For peers
>>    with large out-degree, this could be likely.  Were these
>>    data interesting in any way, or just more storage?

Yes, you presume correct. I honestly didn't find that really useful... A simple "Peer down" message would have been sufficient in those cases to replace the withdrawn storm, while the announcement storm caused by the RIB transfer of peers was inevitable.

>>  again, thanks so much for real experience.  gives me a bit more
>>  clue.

You're most welcome!

Best Regards,

Alessandro Improta
Engineering manager
p. +393488077654
e. aimprota at catchpoint.com<mailto:aimprota at catchpoint.com>
a. Via Oberdan 53, Pietrasanta (LU)
[cid:64b79133-76ac-486c-8aea-c2984ec6f4ab]
Learn more about Catchpoint → Watch this 2-minute video!<https://www.catchpoint.com/explainer>
[linkedin]<https://www.linkedin.com/company/catchpoint-systems-inc>             [twitter] <https://twitter.com/Catchpoint>              [facebook] <https://www.facebook.com/catchpoint/>               [youtube] <https://www.youtube.com/c/Catchpoint/>


________________________________
From: Randy Bush <randy at psg.com>
Sent: Friday, March 17, 2023 5:15 PM
To: Alessandro Improta <aimprota at catchpoint.com>
Cc: Measurement Analysis and Tools Working Group <mat-wg at ripe.net>
Subject: Re: [mat-wg] some thoughts on add-path and bmp at ripe/ris

alessandro,

real data and experience deeply appreciated.  a few questions.

  o once a collector commits to ADD-PATH, it's a promise to keep
    the data forever; well, it's been 26 years of RV and a few
    less for RIS.

    - Did operators find the ADD-PATH data sufficiently useful
      to be worth the cost?  How did they use the ADD-PATH data
      and how often?

    - Did researchers find the ADD-PATH data sufficiently useful
      to be worth the cost?  How did they use the ADD-PATH data
      and how often?

    - Did folk develop special tools to take advantage of
      ADD-PATH data?

  o On the problem of stream/peer identification.  You describe
    the divergence of BIRD and FRR.  What about commercial
    router vendors?

  o You say data were often redundant, though not fully.  Did
    you investigate mechanisms to reduce the storage, or did you
    see that as a path to complexity and fragility merely to
    save some spinning rust?

  o You mention the withdraw storm between your peer and their
    peer (and later an announcment storm, I presume).  For peers
    with large out-degree, this could be likely.  Were these
    data interesting in any way, or just more storage?

again, thanks so much for real experience.  gives me a bit more
clue.

randy
-------------- next part --------------
An HTML attachment was scrubbed...
URL: </ripe/mail/archives/mat-wg/attachments/20230317/9c336aa6/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Outlook-ajnjxisv.png
Type: image/png
Size: 7788 bytes
Desc: Outlook-ajnjxisv.png
URL: </ripe/mail/archives/mat-wg/attachments/20230317/9c336aa6/attachment-0001.png>