This archive is retained to ensure existing URLs remain functional. It will not contain any emails sent to this mailing list after July 1, 2024. For all messages, including those sent before and after this date, please visit the new location of the archive at https://mailman.ripe.net/archives/list/ripe-atlas@ripe.net/
[atlas] Inconsistencies in anchor API and measurements
- Previous message (by thread): [atlas] RIPE measurement Credits required
- Next message (by thread): [atlas] Inconsistencies in anchor API and measurements
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
Malte Appel
malte at iij-ii.co.jp
Mon Nov 15 08:32:44 CET 2021
Hi everyone, sorry in advance for the long mail. tl;dr: Anchor API and UI give inconsistent results. Some anchor mesh measurements could be fixed, some might target non-anchors. Not sure what to do about it. I am currently working a lot with RIPE Atlas and recently wanted to use the anchors and their mesh measurements in particular. I wanted to answer two simple queries: 1. Get a list of all active anchors 2. Get a list of all active anchor mesh and probe measurements (traceroute, for my particular use case) However, while trying to answer these queries, I stumbled upon quite some inconsistencies, depending on which interface / API is used. As far as I can tell there are four ways in which one could technically answer query 1: 1. Look at the website: https://atlas.ripe.net/anchors/list/ -> 840 results 2. Query the /anchors API with attribute is_disabled = false: https://atlas.ripe.net/api/v2/anchors/?is_disabled=false -> 844 results 3. Query the /probes API with attributes is_anchor = true and status = 1 (Connected): https://atlas.ripe.net/api/v2/probes/?is_anchor=True&status=1 -> 729 results 4. Query the /probes API with attributes tags = system-anchor and status = 1 (Connected): https://atlas.ripe.net/api/v2/probes/?tags=system-anchor&status=1 -> 729 results Methods 3 and 4 are actually consistent! The main discrepancy is between the /anchors and /probes API: 116 anchors that are listed on the webpage and/or /anchors API as ‘active’ are inactive (97 abandoned; 19 disconnected at time of writing) according to their respective /probes entry. I understand that disconnects might be temporary, but some anchors seem to be inactive for years (at least according to their status) and are still listed as active. I have attached a text file with some notes that go deeper into the differences, but might be hard to read. For query 2, I faced a similar situation: 1. Look at the website: https://atlas.ripe.net/anchors/list/full/ Anchoring Mesh IPv4: 840 Anchoring Mesh IPv6: 739 Anchoring Probes IPv4: 840 Anchoring Probes IPv6: 705 Total: 3124 2. Query the /anchor-measurements API Anchoring Mesh IPv4: 849 Anchoring Mesh IPv6: 743 Anchoring Probes IPv4: 902 Anchoring Probes IPv6: 753 Total: 3247 3. Query the /measurements API with attributes status = 2 (Ongoing), type = traceroute and corresponding attributes: Anchoring Mesh IPv4: af = 4; tags=anchoring,mesh Anchoring Mesh IPv6: af = 6; tags=anchoring,mesh Anchoring Probes IPv4: af = 4; tags=anchoring,probes Anchoring Probes IPv6: af = 6; tags=anchoring,probes For example: https://atlas.ripe.net/api/v2/measurements/?status=2&type=traceroute&af=6&tags=anchoring,probes Anchoring Mesh IPv4: 1026 Anchoring Mesh IPv6: 902 Anchoring Probes IPv4: 668 Anchoring Probes IPv6: 619 Total: 3215 These results are even more mixed: - Tags can be inconsistent: Some measurements have none, some have the ‘probes’ or ’mesh’ tag, but miss the ’anchoring’ tag. - Some anchors have multiple measurements (especially probes measurements), of which most actually are run by the same set of probes, i.e., they are duplicates. - Which measurements are contained in which of the three result sets is very mixed, maybe I should draw a Venn diagram :) Finally, I looked at the consistency of the IP addresses of the /anchors API (ip_v4 and ip_v6), the /probes API (address_v4, address_v6), the DNS result for the FQDN of the anchors, and the target IP of the mesh/probes measurements. I noticed some problems, since our lab (IIJ) also operates an anchor (probe 6425 [0]) and we updated the IP address some time ago, but are actually not reached by the mesh measurement, because the measurement still targets the old IP. I attached a CSV that includes the raw data (of measurements with some form of problem), but basically there are 93 measurements from connected anchors that fail, and out of which 68 (from 29 anchors) could work, if the measurement would target the correct IP. These measurements have matching anchor/probe IPs and DNS records, so I do not know why the measurement target is stale. There are some additional measurements that could work, but it is unclear what the intended ‘correct’ IP is. On that note, there are 48 measurements that ‘work‘, i.e., they get a response from the target, but it is not clear if the target is the intended receiver: - 8 target abandoned anchors - 18 have different probe and anchor IPs and target one of them - 21 have the same probe and anchor IP but target something else Again, I am sorry for this long mail. I understand that RIPE Atlas is a huge project that has grown over time so it might be hard to keep some things synchronized, and some other things might not be easily decidable (e.g., when to mark an anchor is inactive). However, I think especially the IP address of an anchor in the /anchors and /probes APIs, in the DNS entry, and the target of the mesh/probes measurements need to be consistent. Currently some mesh measurements might target an entirely different machine. I wanted to bring some attention to this, but not sure what else I can do as a user. I don‘t want to complain too much :) For now I will just use all data sources as input and apply some sanity checks. Best, Malte P.S.: Some feedback on how we can bring the measurement of our anchor to target our anchor would be nice though. [0] https://atlas.ripe.net/probes/6425 -------------- next part -------------- A non-text attachment was scrubbed... Name: wrong_ip_msm.csv Type: text/csv Size: 56448 bytes Desc: not available URL: </ripe/mail/archives/ripe-atlas/attachments/20211115/2e40a63c/attachment.csv> -------------- next part -------------- Four ways to get list of active anchors: 1. Look at the website: https://atlas.ripe.net/anchors/list/ -> 840 results 2. Query the /anchors API with attribute is_disabled = false: https://atlas.ripe.net/api/v2/anchors/?is_disabled=false -> 844 results 3. Query the /probes API with attributes is_anchor = true and status = 1 (Connected): https://atlas.ripe.net/api/v2/probes/?is_anchor=True&status=1 -> 729 results 4. Query the /probes API with attributes tags = system-anchor and status = 1 (Connected): https://atlas.ripe.net/api/v2/probes/?tags=system-anchor&status=1 -> 729 results 3 and 4 are consistent! Unique to 2: Probe 6288 Hostname: us-pdp-as15133 Anchor ID: 1063 Abandoned since 2020-09-21T23:34:29Z Both /probes address_v4 field and /anchors ip_v4 field point to 152.195.252.99 Anchoring Mesh ping and traceroute measurements go to 192.229.156.101, which seems to be an old IP according to the changelog: 2018-12-10 15:18 UTC IPv4 address changed from (u'192.229.156.101', u'152.195.252.35') to 152.195.252.35 Both addresses still reply. Probe 6399 Hostname: nl-ams-as20857 Anchor ID: 1411 Abandoned since 2020-04-28T09:18:55Z Definitely dead, ping and traceroute not responding. Unique to 3/4: Probe 6890 Hostname: ca-van-as395152 Listed as “decommissioned as of 2020-11-09 09:21 UTC”, but still has is_anchor = true Probes and webpage, but not API: Probe 7045 Freshly setup anchor, will probably appear in API at some point. Missing from webpage, but present in API and probes: Probe 6959 Hostname: at-hit-as34347 Anchor ID: 2762 Active and responding, no idea why not listed. Probe 6363 Hostname: fr-ysd-as29169 Anchor ID: 1163 Again, IP change to 217.70.176.109, measurements point to 217.70.181.69, not responding there. Main problem: 116 anchors listed on the webpage and/or /anchors API, but not active in /probes: Abandoned: 97 16 reachable: 6176, 6204, 6215, 6259, 6288, 6385, 6397, 6459, 6481, 6529, 6555, 6601, 6720, 6735, 6791, 6976 Disconnected: 19 2 actually reachable: 6907, 6928 6907 uses wrong target IP for measurements: Resolved to {103.122.188.7, 103.122.190.141}, uses 103.122.188.7, but should now use 103.122.190.141. All reachable anchors are listed in the API and website (except 6288; API only) 98 dead anchors. Getting active anchoring mesh and probes measurements (traceroute only): 1. Look at the website: https://atlas.ripe.net/anchors/list/full/ Anchoring Mesh IPv4: 840 Anchoring Mesh IPv6: 739 Anchoring Probes IPv4: 840 Anchoring Probes IPv6: 705 Total: 3124 2. Query the /anchor-measurements API Anchoring Mesh IPv4: 849 Anchoring Mesh IPv6: 743 Anchoring Probes IPv4: 902 Anchoring Probes IPv6: 753 Total: 3247 3. Query the /measurements API with attributes status = 2 (Ongoing), type = traceroute and corresponding attributes: Anchoring Mesh IPv4: af = 4; tags=anchoring,mesh https://atlas.ripe.net/api/v2/measurements/?status=2&type=traceroute&af=4&tags=anchoring,mesh Anchoring Mesh IPv6: af = 6; tags=anchoring,mesh https://atlas.ripe.net/api/v2/measurements/?status=2&type=traceroute&af=6&tags=anchoring,mesh Anchoring Probes IPv4: af = 4; tags=anchoring,probes https://atlas.ripe.net/api/v2/measurements/?status=2&type=traceroute&af=4&tags=anchoring,probes Anchoring Probes IPv6: af = 6; tags=anchoring,probes https://atlas.ripe.net/api/v2/measurements/?status=2&type=traceroute&af=6&tags=anchoring,probes Anchoring Mesh IPv4: 1026 Anchoring Mesh IPv6: 902 Anchoring Probes IPv4: 668 Anchoring Probes IPv6: 619 Total: 3215 Unique to 2: Measurement IDs: 16442292, 16442295, 16442306, 16442309 Mesh IPv4, IPv6, Probes IPv4, IPv6 for nl-ams-as20857.anchors.atlas.ripe.net Anchor ID: 1411 Probe ID: 6399 Abandoned since 2020-04-28T09:18:55Z Unique to 3: 82 Measurements All from 1 are in 2 1 but not 3: 110 Measurements 2 but not 1: 123 Measurements 2 but not 3: 114 Measurements 3 but not 1: 201 Measurements 3 but not 2: 82 Measurements From 2: # Tag Consistency 76 measurements without ‘anchoring’ tag, but with ‘probes’ or ‘mesh’ tag 38 measurements without ‘anchoring’, ‘probes’, or ‘mesh’ tag # Multiple measurements per anchor 84 anchors / 103 measurements with multiple mesh and/or probes measurements. 3 Mesh IPv4 56 Probes IPv4 44 Probes IPv6 82 anchors / 100 measurements with same participating probes. 3 Mesh IPv4 Anchor ID: 2082 Measurement IDs: 24433722, 24433725 Anchor down Anchor ID: 2214 Measurement IDs: 25155868, 25155871 Anchor down Anchor ID: 2572 Measurement IDs: 28329447, 28329450 55 Probes IPv4 42 Probes IPv6 2 anchors / 3 measurements with different participating probes. 1 Probes IPv4: Anchor ID: 2748 Measurement IDs: 29622276, 29622279 2 Probes IPv6: Anchor ID: 2748 Measurement IDs: 29622287, 29622290 Anchor ID: 2761 Measurement IDs: 30624813, 30624816 # Measurements for non-anchors Anchoring mesh and probes measurements for two anchors without /anchors API entry, and generally without any entry. Anchor ID: 1171 Probe ID: 6356 Hostname: au-syd-as23719 Disconnected since 2021-09-02T00:46:30Z Mesh: Anchor Measurement ID: 4095 Measurement ID: 12237261 Probes: Anchor Measurement ID: 4098 Measurement ID: 12237268 Anchor ID: 1600 Probe ID: 6500 Hostname: ir-thr-as8868 Abandoned since 2019-08-27T06:57:40Z Mesh: Anchor Measurement ID: 5824 Measurement ID: 19508319 Probes: Anchor Measurement ID: 5827 Measurement ID: 19508349 Failing Atlas measurements from /anchor-measurements API Out of 3247 measurements 166 measurements fail 70 Abandoned 6 (from 2 anchors) would work if their target IP would be updated 3 Disconnected 1 (from 1 anchor) would work if the target IP would be updated 93 Connected 68 (from 29 anchors) would work if their target IP would be updated (probe IP == anchor IP == DNS entry) 2 (from 1 anchor) probe == DNS, works; target == anchor, fails 2 (from 1 anchor) probe != anchor != target; DNS == anchor; both probe and anchor would work. 8 (from 3 anchors) probe works; anchor == target == DNS, fails 1 (from 1 anchor) has wrong target; probe == anchor that works, but no DNS entry? *1) 3 (from 1 anchor) probe == anchor, works; dns == target, fails 84 measurements could be ‘resurrected’ 9 are borked (none of the IPs respond) 2 (from 1 anchor) target == anchor == dns, fails; probe has no IPv4 anymore 48 work with problems 8 (from 4 anchors) work, even though anchors are Abandoned -> One-way 6 (from 3 anchors) have no DNS entry anymore 2 (from 1 anchor) have a different target IP, but probe == anchor; both work 2 (from 1 anchors) target == probe; dns == anchor; both work 18 (from 8 anchors) have different probe and anchor IPs 2 (from 1 anchor) target probe (for these the anchor IP does not work) 16 (from 7 anchors) target anchor (the probe IP would also work) 2 (from 1 anchor) resolve DNS to probe, i.e., target does not match DNS 21 (from 11 anchors) have the same probe and anchor IP, but target something else 19 (from 10 anchors) target and probe/anchor work 2 (from 1 anchor) target works, but probe/anchor does not 1 (from 1 anchor) has no DNS entry *1) The same. Different targets for Probes (this one; works) and Mesh (the one above; fails) -------------- next part -------------- A non-text attachment was scrubbed... Name: OpenPGP_0x877CEAFF9EA25E92.asc Type: application/pgp-keys Size: 3138 bytes Desc: OpenPGP public key URL: </ripe/mail/archives/ripe-atlas/attachments/20211115/2e40a63c/attachment.bin> -------------- next part -------------- A non-text attachment was scrubbed... Name: OpenPGP_signature Type: application/pgp-signature Size: 840 bytes Desc: OpenPGP digital signature URL: </ripe/mail/archives/ripe-atlas/attachments/20211115/2e40a63c/attachment.sig>
- Previous message (by thread): [atlas] RIPE measurement Credits required
- Next message (by thread): [atlas] Inconsistencies in anchor API and measurements
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]