[mat-wg] Querying popular domains with RIPE Atlas

Dear colleagues,

As you are probably aware, DNSMON* and RIPE Atlas built-in**
measurements are often used when analysing outages and other issues with
important DNS services, particularly the root zone servers. These
measurements do not provide a direct indication of the impact on actual
end-user experience because they involve directly querying authoritative
name servers, thereby ignoring the effects of having multiple
authoritative name servers as well as caching local recursive resolvers.

To improve upon our DNS monitoring, we will initiate two built-in DNS
measurements from all RIPE Atlas probes that use the probes' default
resolvers instead of authoritative name servers. The first measurement
will query for random top-level domains with the intention of avoiding
caches and therefore hitting at least one root name server. The second
measurement will query for popular domain names, with the intention of
hitting caches where appropriate and getting an idea of the true impact
on users.

As a starting point for the second measurement, we will use a recent
snapshot of the global top 50 visited sites according to the Alexa top
sites list***, which combines unique visitors and individual page views
in its calculations. Probes will cycle through these domains, querying
for a different A record every 10 minutes.

Since DNS queries containing these domains will be made from every
connected RIPE Atlas probe, we would like to ask for feedback on whether
querying the DNS for any of these domains could pose legal or political
problems for probe hosts in certain parts of the world. I have also
included the next 25 most popular domains below, which will be used to
fill any gaps caused by removing items from the main list.

Kind regards,
Chris Amin
RIPE NCC

* https://atlas.ripe.net/dnsmon/
** https://atlas.ripe.net/docs/built-in/
***
https://en.wikipedia.org/w/index.php?title=List_of_most_popular_websites&oldid=762455912

Provisional 50 domains to query
===============================
google.com.
youtube.com.
facebook.com.
baidu.com.
wikipedia.org.
yahoo.com.
google.co.in.
amazon.com.
qq.com.
google.co.jp.
live.com.
taobao.com.
vk.com.
twitter.com.
instagram.com.
hao123.com.
sohu.com.
sina.com.cn.
reddit.com.
google.de.
linkedin.com.
tmall.com.
yahoo.co.jp.
weibo.com.
google.fr.
360.cn.
google.co.uk.
google.ru.
google.com.br.
yandex.ru.
ebay.com.
bing.com.
msn.com.
google.it.
soso.com.
t.co.
wordpress.com.
google.es.
microsoft.com.
tumblr.com.
aliexpress.com.
blogspot.com.
netflix.com.
amazon.co.jp.
ok.ru.
google.com.hk.
stackoverflow.com.
google.ca.
google.com.mx.
imgur.com.

Next 25 most popular domains, excluding pornography
===================================================
apple.com.
Naver.com.
mail.ru.
imdb.com.
popads.net.
onclickads.net.
office.com.
google.co.kr.
github.com.
pinterest.com.
paypal.com.
tianya.cn.
google.com.tw.
google.com.tr.
google.com.au.
diply.com.
amazon.de.
google.co.id.
microsoftonline.com.
onclckds.com.
twitch.tv.
adobe.com.
amazon.co.uk.
wikia.com.
cnzz.com.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: </ripe/mail/archives/mat-wg/attachments/20170207/78c8b19f/attachment.sig>