[atlas] Making a bunch of tests/measurements to a single destination is 'tedious'

On Thu, Jan 24, 2019 at 3:52 AM Shane Kerr <shane at time-travellers.org>
wrote:

> Chris,
>
> On 24/01/2019 06.39, Christopher Morrow wrote:
> > howdy!
> > So, if I have a dns server somewhere and I want to make a bunch of
> > measurements:
> >    "how does NYC see my dns?" (pick N probes near NYC)
> >    "How does DEN see my dns?" (pick N probes near DEN)
> >    "How does LHR see my dns?" (pick N probes near LHR)
> >    "how does ...."
> >
> > ...you get the idea... I'm trying to characterize my service from a
> > 'user' perspective by large Metropolis, well within N km of that
> > metropolis anyway :) but across ~70 or so metros, in v4 and v6 and with
> > authoritative and non-authoritative queries with each test running for 4
> > days so i can see some daily cycles in traffic patterns/etc. The number
> > of measurements is large in total, breaking up by 3-4 metropolis chunks
> > is super tedious :(
> >
> > Can I request an API key (or some other thing) which can make that sort
> > of request happen for all my measurements in one go? The thing I'm
> > probing is very able to handle a few extra thousand queries per second..
> > and I'd only be hurting myself i get my estimate wrong :)
>
> I was part of a team on a hackathon a while back which needed to compute
> distances between points on the globe, including Atlas probes:
>
> https://github.com/shane-kerr/ripe-atlas-anycast-work
>
> I see that the openflights project has since moved to GitHub, and so you
> can get the IATA airport data here:
>
> https://github.com/jpatokal/openflights/blob/master/data/airports.dat
>
>
The geo data wasn't problematic for me, once I found:
  github.com/kelvins/geocoder

which apparently uses:

https://raw.githubusercontent.com/jpatokal/openflights/master/data/airports.dat

oh hai! It looks like I didn't do it the 'insane way', mostly :)
(I should make the repository of code I wrote available, oops)

I guess I should update the README.md...
>
> The repository is basically a last-point-in-time snapshot of the
> hackathon work, so it a bit rough. 😬 Anyway, this code is probably the
> closest to what you want:
>
>
> https://github.com/shane-kerr/ripe-atlas-anycast-work/blob/master/add-dist.py
>
>
i think you are just sorting through measurement response data to find
things near/around/at your points of interest here, right?
that seems useful.


> Turning the great_circle_dist() function into a filter across the set of
> all probes (meta-probes.json, which you can download in advance) should
> get you something like what you want.
>
>
looks neat ;)


> Note that this was all done 4 years ago according to GitHub, so maybe
> Atlas provides better tools for this nowadays. 😊
>
>
they do have a method to request probes based on geo data and a radius from
that point, which is what I am doing.
They also have filters (which I've apparently sucked at reading
completely!) to limit based on type/ip-version/activity/etc.
mostly you can query for: "probes near here with these sets of attributes"
which works pretty well ;)

my main problem was/is (I think) that I want to say:
  "For these M metros, please find 10 (or so) probes with working address
family (v4||v6) and probe dns server (V4||V6 address) with a
(non-)authoritative request"

my M number is ~70+.
I didn't want (for some reason?) to try and gather the probes across the
globe into less measurements but more probes... huh, I could try that I
guess, then I'd only have 4 measurements not M*4.

Cheers,
>
> --
> Shane
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: </ripe/mail/archives/ripe-atlas/attachments/20190124/908ef9a8/attachment.html>