This archive is retained to ensure existing URLs remain functional. It will not contain any emails sent to this mailing list after July 1, 2024. For all messages, including those sent before and after this date, please visit the new location of the archive at https://mailman.ripe.net/archives/list/ripe-atlas@ripe.net/
[atlas] Planned changes to "measurement archive"
- Previous message (by thread): [atlas] Extreme slowness today?
- Next message (by thread): [atlas] How can I reduce data usage on remote satellite-connected islands?
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
Robert Kisteleki
robert at ripe.net
Wed Sep 9 11:48:32 CEST 2020
Dear RIPE Atlas users, You may be aware that we publish "measurement metadata dumps" on "FTP" (https://ftp.ripe.net/ripe/atlas/measurements/) every week. These are meant to make the complete measurement metadata downloadable for those who are interested. Even though there is still a point in publishing our metadata in a downloadable format, the current method is not sustainable: as it is defined it'll grow forever - which is not realistic. Currently *every* dump contains *all* the measurements we *ever* had in the system. We plan to change that to "everything that's running now + everything that was running in the last two weeks". This has about 200k items currently. If we published this every day instead of once a week, we will still only produce 10% of the output we do today - by virtue of eliminating 90% of redundancy and repeated work. The syntax and semantics of the content will not change. We will keep the existing dumps as they are mostly because there's no reason to remove them. What this means for users is: * if you care about "current / recent" measurements, you can just use the last dump from yesterday * if you really need all (or a large part of) history, you can fetch one dump per 2 weeks or so and combine them * you can still use the API for random, finite searches We plan to implement the above change just after RIPE 81 (i.e. early November). Please be aware of this change if you rely on these dumps. Regards, Robert
- Previous message (by thread): [atlas] Extreme slowness today?
- Next message (by thread): [atlas] How can I reduce data usage on remote satellite-connected islands?
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]