RIPE 64

Minutes from RIPE 64
DNS Working Group
Wednesday 18 April 2012
Ljubljana, Slovenia

RIPE 64 -- DNS Working Group -- Session I

0. Administrivia

Two of the DNS Working Group chairs, Jaap Akkerhuis and Jim Reid, opened the first session of the day. They introduced the Working Group and explained that the meeting was being recorded, reminding attendees to use the microphones for asking questions.

Jim Reid brought up the minutes of the Working Group from the previous RIPE Meeting, asking the attendees for acceptance or comments. There were no comments and the minutes were approved.

1. Robert Kisteleki, RIPE NCC -- RIPE NCC Report

https://ripe64.ripe.net/presentations/36-RIPE64_DNS_Update.pdf

Marco Davids, SIDN asked about the criteria for running slave servers for TLDs, specifically ICANN's new gTLDs.

Daniel Karrenberg, RIPE NCC, explained that the RIPE NCC started doing that when many ccTLDs in emerging countries were in their infancy. The intention was to help those emerging countries. He added that now the situation is different and the RIPE NCC membership didn't think they should pay for secondary services for these sometimes very large TLDs. It was decided that the RIPE NCC would not offer DNS service to new gTLDs and if the community felt different they should say so in the RIPE NCC Services Working Group.

Marco said he had the same understanding but he had been surprised by the number of remaining TLDs.

Daniel replied that the vast majority were very small ccTLDs and that a more detailed list could be provided.

Olaf Kolkman, NLnet Labs, asked that the statistics about DNS key queries to the root servers be published in the next three reports because they gave an indication of the amount of deployment of DNSSEC and validating resolvers in the world. He added that the stats could be interpreted very differently and he wanted to see if there was any development there.

Robert replied that the details would be provided in the next update.

2. Benjamin Zwittnig, ARNES -- DNSSEC in .si

https://ripe64.ripe.net/presentations/76-BZRIPE1.pdf

Jim Reid asked about management of the signing key and the key repository.

Benjamin replied that the key was stored in the HSM and they had backed it up during the key signing ceremony.

Peter Koch, DENIC asked whether they were a happy customer of the Sun SEA 6000 and whether they had any succession plans for that technology.

Benjamin replied they weren't that happy, they had found the cards useful but would probably look into replacing them in the future. He added they only used them for the KSKs.

Peter Koch asked the attendees if there were other people in the room with the same piece of hardware in production that maybe would like to get together to find a replacement for it.

A member of the audience mentioned they were having the same issue as Peter, the hardware had been discontinued because it was not Sun hardware. However they were now using cards from Talis (Thales? -JR): slightly more expensive but with a stable support chain.

Lars Liman from Netnod asked what the process was for a domain holder in a Slovenian TLD to have his zone signed or connected to the parent zone.

Benjamin replied that the process was manual, and registrants would have to contact ARNES instead of a registrar.

3. Willem Toorop, NLnetLabs -- DNSSexy

https://ripe64.ripe.net/presentations/116-dnssexy-Willem-Toorop.pdf

Jakob Schlyter, Kirei AB said they had discontinued their auditor and it would not be part of OpenDNSSEC any more, therefore he found the project quite interesting. He asked whether it would be integrated back into NSD or if it was a fork, and in that case if changes would be ported to DNSSEXY.

Willem answered that the fixes from NSD would be incorporated into DNSSEXY, but that it was indeed a fork and would remain separate from NSD.

4. Patrick Wallström, IIS -- Quality of DNS and DNSSEC in the .se Zone

https://ripe64.ripe.net/presentations/87-Patrik_Wallstrom_RIPE64_2012-04-18_DNSSEC_in_.SE.pdf

Peter Koch, DENIC, enquired about the signature lifetimes, whether the KSK signatures were done over the DNS key or the DSK signatures over some random RRset.

Patrick said it was random, so it included both. He had found it to be always the same without exceptions.

Peter asked about the registrars or nameserver providers with a large number of domains, as that could make the statistics biased and whether the research did anything to mitigate that.

Patrick answered that it depended on how it was done, for example if the defaults for the DNSSEC signer were always used and not changed when starting DNSSEC.

Peter commented that if such an entity with many zones decided to use random sized ZSKs, they would influence the results dramatically. He compared the situation to looking for a "swarm" or distributed intelligence. He questioned how much they were judging what people were doing or if they were all following everyone else, waiting for a someone else to take a different approach to key lengths and rotation policies.

A remote participant, Gilles Massen, DNS.LU, asked whether the statistics had been compared to any other zones.

Patrick replied that it was something that could be done quickly, and he would do it.

A member of the audience stressed the need for better recommendations, perhaps from the RIPE DNS Working Group. He noted that some vendors shipped devices that had poor defaults, and people tended to either use those values or copy them from somewhere else. He asked the Working Group to take up the work of creating such recommendations.

Jim Reid asked the audience speaker if they were volunteering to write that document.

The audience member replied that he didn't say that, but he would be willing to work on it.

George Michaelson, APNIC, commented that from the slides it was apparent that 6% of the zones were bad, because about 1 in 15 questions failed, and that it showed nobody was doing validation.

Patrick replied that all major ISPs in Sweden were doing validations, however the domains in question were not taken into account since they were not used for any purpose.

George commented that meant those were faulty signed domains but since nobody was looking at them there was no perception of a problem. He added that key lengths kept getting bigger, and nobody seemed to question that. He wondered if a future of 4,000-bit long ZSKs was really necessary or perhaps 512-bit keys could be sufficient for the duration of a suitably short key life time.

Olaf Kolkman from NLNetLabs replied that he was taking part in another forum what is trying to come up with recommendations for that. The document was currently in its second iteration (RFC 4641 BIS) at the IETF WG. He added that getting recommendations out was a wise thing to do, but not an easy task and difficult to get consensus on.

Peter Koch commented that he was involved in the forum that Olaf had mentioned and author of RIPE 203 at the same time. He said that while it was possible to get consensus on a trade-off document, it was close to impossible to reach consensus on concrete figures for recommendations. He added that the target audience was difficult to pin down and that finding a sensible target audience would be the first step before even thinking about writing the document.

Patrick said that when one authors tools such as DNSSEC or a zone checker, it would verify the same zone configuration. He appreciated the RIPE Document as being short enough to point to while the RFCs were too many and there was basically no direct recommendation in there. That made them very hard to use for design choices about validation errors or warnings.

Peter said he was aware of the split and one of the reasons was explaining the trade-offs between different choices. He warned that the crisp nature of RIPE 203 had the downside of being easily misapplied to the wrong target.

Randy Bush, IIJ commented that 15 years ago they had built something very complicated which crashed regularly. There was a shortage of people with the required technical knowledge to control it. He proposed to simplify the architecture and document it properly before adding more features to it.

5. Roland M. van Rijswijk, SURFnet Middleware Services -- DNSSEC: Dealing With Hosts That Don't Get Fragments

https://ripe64.ripe.net/presentations/91-20120418_-_RIPE64_-_Ljubljana_-_DNSSEC_-_UDP_issues.pdf

Jakob Schylter asked what other services broke before the ones mentioned. He explained that if the resolver operators had a problem with broken firewalls and similar, those problems should exist even before starting any work on the DNS side and it would break before hitting the troublesome SURFnet servers.

Roland answered that answers for the root zone had been optimised and they were much smaller. The issues would not be encountered there, and not even with some of the TLD names. He added that for longer names the answers would be longer as well and the zones would become larger as there was more data in them, making a tier two domain owner the first to notice such a problem.

Jakob commented that in order to fix the issue, a big domain name would have to be DNSSEC signed. Roland suggested "mozilla.org".

Edward Lewis from Neustar pointed out a mistake on one of the slides and Roland said it would be fixed.

Lars Liman, a root server operator added that there ought to be recommendations for firewalls as well.

Roland agreed and added that perhaps software vendors should be added to the list as well.

João Damas, ISC, commented that the failure rates when IPv6 was introduced were a lot lower than the failure rates in this case. However the attitude of the people involved was different: nobody told users that they were wrong and that they should fix their own equipment lest they would encounter breakage. He suggested that perhaps it was time to change peoples' attitudes.

Roland replied that regular users do not care about such things and the first step would be for the technical people to make sure the default settings didn't break anything. He added that for DNSSEC that meant starting with recursive resolvers.

João agreed that regular users should not even come into the question since all they wanted was to navigate and communicate. He added that in the case of IPv6 nobody approached the end user directly but their ISPs.

Roland said that in the case of IPv6 the problem was actually much worse since IPv6 firewalls didn't do fragment re-assembly at all, because fragmentation occurred between most end points of a connection and there should have been no fragmentation or re-assembly in between.

6. Follow-up to Plenary DNS Presentations

Jim Reid noted that the Working Group was running over time, but there was time left for follow-up discussions on the two Plenary presentations that related to DNS. He mentioned there was some discussion after Joseph's presentation on route objects and authentication with secure DNS. He asked if anyone had anything to add to that or any questions or comments for Joseph Gersch (who presented about "BGP Route Origin Verification Using Reverse‐DNS"). There were no questions.

Jim then invited Ed Lewis, Neustar, to give a more detailed interpretation of a few slides related to his Plenary presentation about "Looking at TLD DNSSEC Practices". The slides can be found here: https://ripe64.ripe.net/presentations/111-RIPE64DNSWGTLDDNSSEC.pdf

Ed commented about an earlier discussion when "clients" were mentioned and the idea about BCP. He recognised the need to have a BCP, but he also acknowledged the difficulty of coming to an agreement about a specific number. He stressed the need for setting up a document with detailed information that could also have helped him in his work, since compliance with RFCs was difficult as they detailed no requirements.

Jim Reid commented that part of the ICANN process to apply for a gTLD was to detail capabilities, which included plans for the TLD to do DNSSEC. He added that the people involved in requesting the TLDs had no idea what DNSSEC was and they had no clue about the things that needed to be told to or asked of the DNS providers.

Edward replied that the list of RFCs was indeed too extensive for easy compliance and they even contained self-contradictory parts, giving RFC 4641 as an example. He expressed the need for a document that detailed exactly what was needed to aid in compliance.

Jim Reid thanked all the speakers, the RIPE NCC staff and the stenographers. He ended the session inviting attendees back for the second one later during the day.

RIPE 64 -- DNS Working Group -- Session II

Jaap Akkerhuis welcomed the attendees and chaired the session.

7. Peter Janssen, EURid -- YADIFA Update

https://ripe64.ripe.net/presentations/132-RIPE64-YADIFA.pdf
João Damas commented that a remote name server configuration protocol was something ISC had implemented some time ago. However other implementers have provided similar functionality in different way. There had been an effort born out of the community to build what is called a NCP ("Name-server Control Protocol"). He asked Peter if they would be interested in joining.

Peter agreed as they had that as a request from operational people as well. They had an idea about how to approach the solution, and had actually implemented something, but it was still in an "alpha" stage. The intention was to have that become a standard of for all DNS implementations.

Lars Liman asked for clarification regarding Peter's statement that it would become a "standard", whether he meant their idea would become a standard or if there would be a standard for doing it.

Peter answered they would love for that to happen and they considered it a good idea. They had also received feedback that people thought it would be good. He added that it was a community effort and whatever came on top, they would implement it if it had proper support.

Lars asked if the software was IPv6 compliant, both in transport and content. Peter confirmed that it was.

Lars also asked about DNSSEC, and Peter confirmed that it was supported as well together with anycasting.

Robert Martin-Legene, Packet Clearing House, said the project looked promising, and he asked what would happen when one of the transport packets was lost.

Peter answered that the connection was over TCP not UDP which ensured re-transmission. He added that the purpose was to give an operator the chance to check the configuration and its completeness before applying it.

Robert said that in his understanding, if something was wrong in (say) Ukraine, the operator would have to manually pick up on it.

Peter explained that while that was true it only applied to the master server which normally would be located close to home. The slaves would receive new data which would contain a full complete configuration and that would be brought live.

Robert said that all those configuration options could be secured with TSIG.

Peter agreed but said they were not there yet as the code was still under development. He added that they wanted people's opinions before going further.

Robert asked if finally they aimed to trace the TSIG to transfer the clear text over wire.

Peter said the solution was not clearly defined yet and that there was a minimal set of pre-configuration to be done on all name servers to have them talk to each other securely.

Jaap Akkerhuis intervened to stop the questions, noting that all the details were a community effort for further discussion.

8. Ondřej Surý, CZ.NIC -- Knot Update

https://ripe64.ripe.net/presentations/125-KNOT-20120418-OS-RIPE64.pdf

Jim Reid asked what was meant by the statement that the latest version of the code had support for the root zone.

Ondřej explained that versions before 0.8 were not even able to load the root zone.

There were no further questions.

9. Jakob Schlyter, OpenDNSSEC -- OpenDNSSEC Status Update

https://ripe64.ripe.net/presentations/84-ripe64-opendnssec.pdf

There were no questions.

10. Robert Kisteleki, RIPE NCC -- RIPE Atlas Measurements & Tools

https://ripe64.ripe.net/presentations/99-RIPE64_DNS_Atlas.pdf

Jaap Akkerhuis noted that from the maps it appeared that China was active on IPv6. Robert replied that was entirely possible.

A member of the audience asked for clarification regarding DNS queries over TCP and if that was on the wish list. Robert replied that they actually did support it, and serial queries were done on both UDP and TCP.

Roland van Rijswijk of SURFnet asked how RIPE Atlas would make sure that everybody got a slot if they all wanted to do queries on the probes, as the probe could run out of time to do the queries.

Robert explained there was a scheduling algorithm that decided which probes would do which type of measurement. He added that not all probes were expected to participate in all the measurements, but once there were more probes active, hundreds or even more probes could be used at once. He added that they had to make sure the probes were not overloaded in terms of how many measurements they did and at the same time that the destinations did not become overloaded. He said rate limits were employed to avoid turning the RIPE Atlas network into a DDoS botnet.

Robert explained there was a credit system in place which defined a limited number of credits to be spent on measurements. He added that the goal was that any single user of the RIPE Atlas should not be able to overload or claim all the capacity of the system.

A remote question from Gilles Massen, DNS-LU, was relayed. He wanted to know how DNSMON related to RIPE Atlas, especially when it came to User Defined Measurements.

Robert replied that DNSMON was still up and running, and encouraged the attendees to come to the MAT Working Group as Daniel Karrenberg had prepared an update about the future of DNSMON. He added that RIPE Atlas was already able to do almost everything DNSMON could do, and it was likely that it would be used as a successor to DNSMON. He asked all those interested in the details to attend the NCC Services and MAT Working Groups, also the working BoF.

Peter Koch recalled the mention of rate limiting, and asked if they had thought about an opt-out list for user defined measurements.

Robert replied that while they had no considered DNS measurements in particular, it was something they wanted to investigate. He added there would be further discussion needed regarding "opt-in" and "opt-out" on the probes as at the moment there was no way to opt-out of DNS measurements.

Peter asked for the features to be shared since dns.org for example had a "do not probe" list that was set up because of some such measurements being poorly done in the past. Robert replied that they had considered the question from the other side, of probe hosts opting out of a particular type of measurement. He added that this was definitely a feature they were thinking about and would possibly implement in the future, giving users the option to define networks they owned and did not want probed.

11. Dave Knight, ICANN -- Dense Anycast Deployment of DNS Authority Servers

https://ripe64.ripe.net/presentations/135-dknight-lroot-dnswg-ripe64.pdf

Daniel Karrenberg commented that he was looking forward to seeing more talks that discussed practical experiences. He asked the DNS Working Group for feedback on which way they wanted to see the RIPE NCC go with K-root: keeping the current model or moving more towards the direction of L-root.

Daniel asked the presenter how they had addressed the problem of potential inconsistencies in the system. He added that as more servers would run, the likelihood would increase that some would provide different answers for various reasons and that could increase monitoring requirements.

Dave replied that they had monitoring in place to quickly fix anything that broke. He added that so far they had not seen the described problems too often, although that could change when scaling up the number of nodes, but they had technical means to manage that.

Daniel asked if the increased monitoring was manageable, and Dave replied that by that point they thought it was.

Jaap asked Daniel for the best place for the community to provide feedback to the RIPE NCC regarding the K-root server.

Daniel pointed him towards the DNS Working Group mailing list or if they thought it was too detailed for the list they could send it directly to him, Robert Kisteleki or other RIPE NCC staff.

A member of the audience commented they were running an anycast network on a smaller scale, however they saw that traffic didn't always end up in the location they expected it to. He asked the presenter how dynamic their BGP network was set up and if the locations were independently monitored to optimise round-trip times. He also asked whether the topological implication of adding a new node were being considered.

Dave replied that their decision process for where to host a node came down to whether the host could buy a box. They considered that if the host could provide a machine and do configure BGP, the node could be deployed without much further analysis.

He added that for analysis they were mostly looking at traffic and took action as necessary. He added that in contrast to other anycast root servers, all of their nodes were global and that would help as they spread out.

Kostas Zorbadelos, OTE SA, submitted a remote question. He asked if they were planning to follow something like the Ubuntu release management and upgrade all the servers every six months. Dave said that they had not decided yet, but they definitely were not running automatic updates. Instead they would periodically run a manual update on a test machine and then apply that in production. He added that thanks to automation of the installation it was easy to fully re-install a box in half an hour and have it come on-line again.

Dmitry Kohmanyuk, Hostmaster Ltd., said they were already hosting two of the L-root nodes and thinking about adding a third. He asked if the old statistics for each individual nodes could be brought back.

Dave answered that the statistics were available, only organised differently. He said it was possible to view nodes split by region and then separate into the individual results.

Jaap stopped the questions and suggested they could continue the discussion off-line. He thanked the speakers and the attendees for their questions.

12. Panel Discussion on DNSChanger

The following point on the agenda was a Panel to discuss DNSChanger.

Jaap introduced the discussion topic, which involved the DNSChanger worm which had a particular way of operating by changing the affected users' local DNS settings to point to rogue sites. The botnet had been dismantled and in the process the rogue servers were now being operated by the ISC, and the RIPE NCC had also been involved via a Dutch court order related to the case.

He gave the floor to Peter Koch who chaired the discussion. Peter explained that while the topic would be discussed in the DNS Working Group it also touched other areas from security and botnet mitigation to Internet governance. He introduced the panel:

• João Damas, ISC
• Jochem de Ruig, RIPE NCC
• Brian Nisbet, co-chair of the RIPE Anti-Abuse Working Group

To provide context for the discussion Peter suggested Jochem give a short statement and then João could explain the international background.

Jochem gave an overview of the situation from the RIPE NCC side, starting with the initial contact by the FBI and the subsequent Dutch Police order. He added that the RIPE NCC had challenged the State on the order and were in legal proceedings with the State regarding it, awaiting the outcome before a judge, aiming to also get some clarity on the process.

Peter asked if the mentioned date of 8 March was the correct one or if that was the determination date given in the order.

Jochem answered that the order stated 22 March but because of the proceedings and the communication they had reversed the order in January already so the date had passed from the RIPE NCC's side. He added that he was aware the order had been extended in the USA up to 9 July.

Peter asked João to expand on the technical aspects.

João related the sequence of events regarding the FBI investigation and ISC's involvement. He explained that law enforcement contacted ISC once it was realised that if that infrastructure was switched off, millions of infected people would lose all Internet connectivity. That would cause a significant increase in the number of support calls to ISPs and also imply a significant financial impact. João pointed out that the ISC had been involved from the outset, bringing in people with experience in the security world like Barry Green or Paul Vixie. He said that the IP addresses in question could not be operated like that forever, since they didn't belong to the ISC and would eventually be reclaimed by the RIPE NCC, although at that point nobody was certain about the end situation.

Peter asked Brian about his perspective on the issue from his position of Anti-Abuse WG Co-Chair.

Brian explained that the issue had not been discussed at length in the Working Group. From their perspective it was just another way people had used to abuse the network and misuse resources in a way they shouldn't have. He added that what to them was the way law enforcement and the technical community reacted to it, and what came out of that in regards to the freezing of registry information by the RIPE NCC. He said the whole incident showed great co-operation, it was positive but it highlighted a number of areas that touched on DNS, Database, Abuse and a wider range of issues.

Peter summarised the discussion so far, pointing out that they had just finished celebrating 20 years of successful industry self regulation, and the involvement of court orders and police orders was a bit odd from that perspective. He understood the imminent danger had already been dealt with at the time of the court order, and asked how that factored into the decisions. He also asked how successful the process to disinfect machines had been with the extra time granted, and if a similar situation could repeat in the future. He asked Jochem about the policy aspects of that.

Jochem explained that a lesson had been learned regarding different style of communication in the USA compared to the Dutch side. He added that in the future when a police order came that hadn't been stamped by a court, the RIPE NCC would not act in the same way as they had done for this case. Jochem also mentioned the NCC kept the membership informed during the event, and that the RIPE NCC was looking forward to the court case and further developments over the coming year.

Peter asked João about the remaining infected systems and whether ISPs should be concerned because of that. João answered that there had been some success, there was a uniform trend of the number of queries going down, but there was still a lot of work to do on the ISPs side. He wasn't sure whether it would be complete by 9 July, and was leaning towards "no".

Jim Reid asked from a jurisdictional perspective, particularly from law enforcement action, what should their reaction be if another European law enforcement agency were to ask for the same measures on the ISP level, especially if the laws of the Netherlands or EU were not at play. João answered that it was a bridge to cross when it came to it. He detailed that it was not a unilateral thing. There were legal processes, different parties involved. He added that it should be taken as an example of how things were to be done. Brian commented that from his discussions with LEAs it was evident that LEAs did not see this as something doable or workable every time or in every situation.

Randy Bush, IIJ said that with his limited understanding of Dutch Law, he had been told that administrative law did not require court orders. He questioned the quick acquiescence of the RIPE NCC in that case, and expressed his worry about attacks on the RPKI in the Dutch legal system as the DNSchanger example may have set a precedent. Jochem replied that he thought it was a fair point, and also why they were following the legal proceedings. A complexity was the International legal treaty process, according to which the prosecutor in the Netherlands had to make sure the US court order met Dutch law. He could not judge that as he was not a lawyer, but that was the process they followed. Randy said the problem when taken to the parallel of the RPKI was that someone's entire business could be killed in such a situation.

Daniel Karrenberg replied that the RIPE NCC Board and the Managing Director had made it clear that they would not execute such an order immediately in the future, they would insist on judicial review and they would defend the rights of their members. He added that the only exceptions would be extreme circumstances like a life-threatening situation.

Lars Liman steered the discussion back to the original topic of DNSChanger, asking about the option of delaying the responses from the temporary servers more and more in order to see how people reacted. Peter thanked him for pointing out an engineering aspect and asked João for a short answer. João's short answer was "no".

Peter wondered why nobody mentioned DNSSEC as a mitigation strategy then before conceding his comment was meant as a joke. He thanked the panel for their contributions, and passed the microphone to Jaap, acknowledging the fact that the session was overtime.

13. A.O.B.

Jaap asked if there was any other business to discuss. He had one thing to note and that Joe Gersch was doing to demonstrate his system that used the DNS for Secure BCP annoucements, directing people to speak to Joe if they were interested.

Ondřej announced that they had KNOT DNS t-shirts and would be distributing them at the back of the room.

Peter had two final things to add for those that had attended previous sessions and might have noted that something had been different in the current agenda. They had left out the reports from IETF and other venues. He asked everyone present to let the Chairs know if they missed these agenda topics or not. That feedback would decide if these reports should be brought back or be skipped at future WG sessions. They could be skipped the following time as well.

Jaap echoed Peter's reminder, then declared the session adjourned.