[mat-wg] [atlas] Querying popular domains with RIPE Atlas

Dear Chris.
It seems fine from my perspective. The second measurement should
be answered by resolver's cache, being top-resolved names.

But what are you thinking on the first measurement? I'd try to
reduce to a minimun sending random queries to root servers, an
infrastructure frequently abused :( Also for the benefit of
transparency and politeness you could prepend some string on
the form "ripe-atlas-test-measurement.<random>."

Best,

Hugo

On 14:52 07/02, Colin Johnston wrote:
> should be fine from uk perspective, note you need to be careful about dos protection issues when probes behind provider mobile nat
> 
> Colin
> 
> > On 7 Feb 2017, at 14:16, Chris Amin <camin at ripe.net> wrote:
> > 
> > Dear colleagues,
> > 
> > As you are probably aware, DNSMON* and RIPE Atlas built-in**
> > measurements are often used when analysing outages and other issues with
> > important DNS services, particularly the root zone servers. These
> > measurements do not provide a direct indication of the impact on actual
> > end-user experience because they involve directly querying authoritative
> > name servers, thereby ignoring the effects of having multiple
> > authoritative name servers as well as caching local recursive resolvers.
> > 
> > To improve upon our DNS monitoring, we will initiate two built-in DNS
> > measurements from all RIPE Atlas probes that use the probes' default
> > resolvers instead of authoritative name servers. The first measurement
> > will query for random top-level domains with the intention of avoiding
> > caches and therefore hitting at least one root name server. The second
> > measurement will query for popular domain names, with the intention of
> > hitting caches where appropriate and getting an idea of the true impact
> > on users.
> > 
> > As a starting point for the second measurement, we will use a recent
> > snapshot of the global top 50 visited sites according to the Alexa top
> > sites list***, which combines unique visitors and individual page views
> > in its calculations. Probes will cycle through these domains, querying
> > for a different A record every 10 minutes.
> > 
> > Since DNS queries containing these domains will be made from every
> > connected RIPE Atlas probe, we would like to ask for feedback on whether
> > querying the DNS for any of these domains could pose legal or political
> > problems for probe hosts in certain parts of the world. I have also
> > included the next 25 most popular domains below, which will be used to
> > fill any gaps caused by removing items from the main list.
> > 
> > Kind regards,
> > Chris Amin
> > RIPE NCC
> > 
> > * https://atlas.ripe.net/dnsmon/
> > ** https://atlas.ripe.net/docs/built-in/
> > ***
> > https://en.wikipedia.org/w/index.php?title=List_of_most_popular_websites&oldid=762455912
> > 
> > Provisional 50 domains to query
> > ===============================
> > google.com.
> > youtube.com.
> > facebook.com.
> > baidu.com.
> > wikipedia.org.
> > yahoo.com.
> > google.co.in.
> > amazon.com.
> > qq.com.
> > google.co.jp.
> > live.com.
> > taobao.com.
> > vk.com.
> > twitter.com.
> > instagram.com.
> > hao123.com.
> > sohu.com.
> > sina.com.cn.
> > reddit.com.
> > google.de.
> > linkedin.com.
> > tmall.com.
> > yahoo.co.jp.
> > weibo.com.
> > google.fr.
> > 360.cn.
> > google.co.uk.
> > google.ru.
> > google.com.br.
> > yandex.ru.
> > ebay.com.
> > bing.com.
> > msn.com.
> > google.it.
> > soso.com.
> > t.co.
> > wordpress.com.
> > google.es.
> > microsoft.com.
> > tumblr.com.
> > aliexpress.com.
> > blogspot.com.
> > netflix.com.
> > amazon.co.jp.
> > ok.ru.
> > google.com.hk.
> > stackoverflow.com.
> > google.ca.
> > google.com.mx.
> > imgur.com.
> > 
> > Next 25 most popular domains, excluding pornography
> > ===================================================
> > apple.com.
> > Naver.com.
> > mail.ru.
> > imdb.com.
> > popads.net.
> > onclickads.net.
> > office.com.
> > google.co.kr.
> > github.com.
> > pinterest.com.
> > paypal.com.
> > tianya.cn.
> > google.com.tw.
> > google.com.tr.
> > google.com.au.
> > diply.com.
> > amazon.de.
> > google.co.id.
> > microsoftonline.com.
> > onclckds.com.
> > twitch.tv.
> > adobe.com.
> > amazon.co.uk.
> > wikia.com.
> > cnzz.com.
> > 
> 
> 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: not available
URL: </ripe/mail/archives/mat-wg/attachments/20170221/9ebfd399/attachment.sig>