[atlas] Feature-suggestion: "round-robin" probes

On Sat, 05 May 2012 12:22:15 +0200
Philip Homburg <philip.homburg at ripe.net> wrote:

> On 5/4/12 23:17 , Jérôme Benoit wrote:
> > On Fri, 27 Apr 2012 13:09:39 +0200
> > "Marco Davids (SIDN)"<marco.davids at sidn.nl>  wrote:
> >
> >> As SamKnows been mentioned here? There's quite some resemblance
> >> between SamKnows and Atlas, and changes for synergy as well.
> > Yes, there are both a closed-source with a secret roadmap project
> > for no good reasons that force similar project that prefer FOSS
> > licences and development model to re-implement software that
> > perform active and passive measurement primitive in the FOSS
> > fashion and hopefully with a better thought architecture to cope
> > with a wide range of needs and large scale campaign coordination
> > runt on end-users "computers" :)
> >
> >
> I don't know about SamKnows, but for RIPE Atlas, talks contain a lot
> of details about how the system works. To the extent that there is a
> well defined road map, we talk about it.

I'm having hard time to find the roadmap and the talks on the RIPE Atlas
web site as a public user with no account. But I might have not
searched enough. Where are they ? 


> As much as I like open source projects, I really don't see it working 
> for the current Atlas system.

It work very well, we do it since age but we took a very different
approach : 

The measurement agent is designed as a standalone component with that
requirement : 

* Must be portable, that mean all measurements must be implemented
  natively.   
* Datagram-based measurement are divided in four components
    - packets forge 
    - packets injection
    - packets capture
    - packets filter 
Each component work is mapped to a dedicated thread (an Lwt job more
precisely, which bring in a very nice way to manipulate packets in a
asynchronous fashion). 
* Other measurement can be mapped with a plugin dynamically loaded (the
  API is not stable yet)
* A syntax (that is not yet finished) permit to express what a
  measurement will do. The syntax is thought as a way to express
  what the software functionalities are and how each component
  interact with each other. I give a working example just to show
  the idea : 


{"probe":
       // the probe module that will be dynamically loaded 
       {"name":["delay","round_trip","icmp","bin"]

       // send and receive sample definition, the time
       // sequence that the measurement will follow while running 
       ,"send":
               {"seq":
                       // repeat 3 times each 30 min, other keyword are
                       // gamma, uniform, poisson, seq can recurse
                       // under conditions   
                       {"repeat":
                               {"count":3
                               ,"seq":{"periodic":{"period":1800.0}}
                               }
                       }
               }
       ,"recv":
               {"seq":
                       {"repeat":
                               // repeat 2 times each 3 seconds the
                               // the sample measurement
                               {"count":2
                               ,"seq":{"periodic":{"period":3.0}}
                               }
                       }
               }
       ,"parallel":
               [
                       // measurement module specific configuration,
                       // here destination and ICMP type 8 packet size
                       {"mark":null
                       ,"data":
                               {"host":"free.fr"
                               ,"size":16
                               }
                       }
                       ,{"mark":null
                       ,"data":
                               {"host":"google.fr"
                               ,"size":16
                               }
                       }
               ]
       }
// where to send the sample result (file, stdout, URI REStful
API) 
,"sample":
       {"file":"-"
       }
}
  
I do not have enough time to describe the whole design in one mail
but the goal is to permit measurement end to end from the network edge
with user participation on end user "computers" or dedicated hardware,
coordinate them and compute analysis on measurement results.  

> Probe are just too fragile, the whole 
> system is too complex.

? 

I fail to understand this argument. 
Every single piece of software is a complex system, that has never ever
be an argument to not opensource it. 
And for the fragility, probably a design issue of the probe (but I
can't known for sure, no source code to review). 

>  Does releasing the Atlas source benefit the
> RIPE community? I don't know. Fortunately, that is not my decision to
> make.

RIPE call. I think RIPE have made a mistake by not going opensource
since the beginning. 

> There is not a lot of magic in Atlas. It is mostly just hard work to
> get all the details right. 

We're working hard and we have the details very right, one piece
after an other, the measurement agent first :)
For example, the difference between wire time timestamp vs the system
time timestamp and timestamping error calibration on the datagram based
measurement. 

> If you have a dedicated team in an open
> source project, then you should be able to duplicate our work. You
> can always for feedback on any design, or ask how we do things.

I do not know the list of active measurements an Atlas probe cover. 

> 
> One word of warning though: try avoid the second system effect. If
> your system is going to do everything it may never get there.
> 

It's a revamp of an old system that will not be disconnect from your
first user base until your user base will have not understood what
you're trying to achieve and the migration plan is complete. The
release cycle will be thought to cope with scalability issue (we
have a huge user base). Don't worry, we know what we are doing, we're
not a young project, just like your current running system we're
redoing :)  


-- 
Jérôme Benoit aka fraggle
La Météo du Net - http://grenouille.com
OpenPGP Key ID : 9FE9161D
Key fingerprint : 9CA4 0249 AF57 A35B 34B3 AC15 FAA0 CB50 9FE9 161D
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: </ripe/mail/archives/ripe-atlas/attachments/20120505/4c627634/attachment.sig>