RIPE 82

Wednesday, 19 May 2021, 16:00–16:45 and 17:00–17:45 (UTC+2)
Chairs: Bijal Sanghani, Kurt Erik Lindqvist, Rob Evans
Scribes: Olivia Mijnals-Ruimwijk / Evelien Schilder
Status: Final

A. Administrative Matters

Kurtis Lindqvist, WG Chair

The presentation is available at:
https://ripe82.ripe.net/archives/video/582

The minutes from RIPE 81 were approved.

B. RIPE NCC Update

Hans Petter Holen, RIPE NCC

The presentation is available at:
https://ripe82.ripe.net/archives/video/583

Elvis Daniel Velea, v4escrow, referred to slide 11 and commented that the ticketing system may work well for the RIPE NCC, but is a failure for the members. He asked if there are plans to give access to the ticketing system (Zendesk) to members in the LIR Portal. Hans Petter responded that this will be on the roadmap for Q3 or Q4 of 2021. It is something that the RIPE NCC will look into, making the tickets and the history available for members.

Erik Bais, A2B Internet, had a question about the uptime for the various services that the RIPE NCC is providing like K-Root, LIR Portal, RPKI, the office infrastructure. He wants to see transparent operational reporting and KPIs set for these services. He asked for a report at the following RIPE Meetings on how these services are doing, specifically the uptime and what were the problems in the previous period. Hans Petter answered that this is one of the objectives of the Executive Board, defining these services. There is already a list of the services that will be defined and include service description. They will add the KPIs as a sort of measurement. The first step is to do this internally, then share it with the Executive Board. If they are confident with the reporting, they can then report it to the members and the community at the next RIPE Meeting.

Elvis asked for a guaranteed promise that by the end of 2021 to have access to the ticketing system, including a history of conversations for LIRs that someone has or receives later. Hans Petter answered that this is not possible. Due to current developments through agile methodology, they only know the plans for the next sprint. Because they do not know if these plans will be successful, they only have firm plans for the coming quarter but not for the following quarters. As part of membership engagement, they are working on a new plan on how to present the next quarter to the members. Hans Petter acknowledged Elvis' frustration and he wants to get into better planning and will commit to it in the next quarter. He will make plans for the next quarter so they can be agile and take upon new things.

Peter Hesler, Independent, asked if the RIPE NCC is considering the availability of IPv6 for all 3rd party hosted services. For example, Zendesk, Kahoot, and SurveyMonkey are all IPv4-based. Hans Peter answered there is a policy that the services we provide should be v4 based. When it comes to a tool that is not v4 based, or v6 based as well, it may not always be an option. He wants to raise awareness on that internally so this will be taken into consideration. Practically, they will see that some tools will have to live on v4.

Dmitry Kohmanyuk, Hostmaster.UA, asked about third-party data handlers, including “cloudisation”, Zendesk, etc. What is the process of choosing a provider? Any chance of public consultation or tender? Hans Petter answered that they have an elaborate process stating that they will solicit three offers but it is not always possible to do that. They have not discussed going into public tenders or having public consultation on vendor selection. It is a heavy process and he is not sure if that is the right way to go. They need to be more formalised on public consultation on important issues.

C. Operational Update

Felipe Victolla Silveira, Chief Operations Officer, RIPE NCC

The presentation is available at:
https://ripe82.ripe.net/archives/video/584

Gert Doering, Independent, commented that he appreciates the aim of "providing a response within one working day", but as a customer it annoys him that every interaction takes another working day. Felipe answered that they also have other KPIs that are implemented as well. The ultimate goal is to complete the ticket in less time and have a quick full ticket resolution. They do not want just to meet the SLA but also aim to have less time to reply and resolve the ticket.

Rüdiger Volk, Independent, asked if it is right to assume that Felipe's ticket statistics are almost/only LIR services. Is there ticketing support available to track (operational or functional) problem reports? Felipe confirmed that the stats that he shared are mostly for RIPE NCC Services and focused on resource management. There are also other queues like reporting on bugs in the software. The RIPE NCC is familiar with the request to have visibility on different issues, but so far, they have not implemented something like this. If the community is interested, they can look into sharing statistics about the different kinds of tickets as well.

Elvis asked how the RIPE NCC calculates the SLA for having at least 1 response in a working day to each and every request. Felipe answered that if they do not reply within one business day, they report this as a failure. In the past, they would reply to the members saying that they would get back to them. This was messing up the statistics. They changed that and now it takes a bit longer to reply and does not violate the SLAs.

Elvis referred to slide 8 and the figure 80%, commenting that this is not just one ticket but quite a lot and asked if they have the exact number of tickets. Felipe confirmed that the tickets from February (85,7%) are indeed a high number. The total number is 5,100 tickets and they failed at 778 of them. In March (97,6%), they had 6,200 tickets and 137 failed. In April (82,6%), they had 5,600 and 931 failed. Fixing this is of high priority.

Elvis commented that he received a couple of emails on a Friday saying that “because Thursday was a vacation day, my request will be evaluated on Monday”. That reply fixed the stats but did not do anything to fix his request. He then asked what is causing this degradation of service. Felipe answered that there were multiple reasons for this, for example in April it was mostly due to national holidays. They have added a lot of extra steps since then in their due diligence, mostly for European Union sanctions checks. In February 2021, they had a lot of people that were absent due to reasons like sickness, maternity leave and they were understaffed. They are working very hard to fix this.

Wessel Sandkuijl, Prefix Broker BV, referred to slide 6 and asked what type of investigations are made and what initiates these investigations. He then asked why there is a sharp increase over the past few years. Felipe informed Wessel that he had reported on this in the past. There has been an increase in the number of frauds and there is a presentation about this at RIPE 77:

https://ripe77.ripe.net/archives/video/2260/
https://ripe77.ripe.net/presentations/79-OperationalUpdate_ServicesWG_11Oct.pdf

Felipe continued that these investigations cover multiple reasons like fraud, hijacks, internal audits. If they see something suspicious, they look into it on their own initiative. A large number of these extra investigations are due to European Union sanction checks. This is a routine check that they perform when dealing with transfers. If there are doubts whether this member is actually on the sanction list or not, it gets escalated to the investigation team. They then look deeper into the matter.

Kurt Kayser, Independent, asked if "investigations" are just internal or if there are external ones too. Felipe answered that they are triggered internally. He is not familiar with any external requests.

James Kennedy, Liberty Global, asked on the topic of tickets, if the RIPE NCC has considered a chat function for LIRs to quickly resolve easier requests. Felipe answered that they used to have a chat function but the workload was too high and they were unable to manage the queues in the chat. They are looking into providing the chat functionality again so things can be solved right away.

D. RIPE NCC and the Cloud

Kaveh Ranjbar & Razvan Oprea, RIPE NCC

The presentation is available at:
https://ripe82.ripe.net/archives/video/585/

Peter Hessler, DENIC, asked if the RIPE NCC estimated the costs of migrating off of the cloud provider in case it is no longer appropriate for use. Razvan Oprea responded that this was considered and that there is definitely a cost that will depend. Some applications you lift and shift (like a VM), others need to be rearchitected, and others moved to another cloud provider.

Randy Bush, RGnet/Arrcus/IIJ. asked how dependent the RIPE NCC has become on unique features of the cloud provider in their test deployments. Kaveh Ranjbar responded that it is easy to get locked into a single cloud provider but that they always keep multi-cloud in mind. With some, it is easier than others. Razvan mentioned BigQuery as an example, that is hard to match with other providers, so it is on a case by case basis.

Gert Doering, SpaceNet AG, made a request to ensure nothing is ever deployed without IPv6 from day one. Razvan responded that they have been pushing cloud providers like AWS in supporting IPv6 in some of their services and they are also re-engineering some internal services to support IPv6. So it is a point they consider fully.

Nicola von Thadden, Pfalzkom GmbH, asked if they have a backup plan in case they are forbidden to store personal data with US cloud providers. Kaveh responded that they definitely have a plan for that but that this is currently not applicable. The mentioned services do not have personal data. They will make sure to follow all the relevant GDPR and other related rules.

Sander Steffann, 6connect, had two questions/comments. The first one was that some services have their primary in the cloud and secondary in house (like the RIPE Database). He thinks that is the wrong way around. Kaveh responded that they will of course still own and are in control of the data. Sander jumped in to say that the blog explicitly said that the read/write copy would be in the cloud and that the RIPE NCC would hold the read-only copy. He would like to see it the other way around. Felipe Victolla Silveira then responded that capacity in AWS (cloud) will be superior to the capacity they currently have in-house and that is the main reason why the RIPE NCC wants to do it this way. Sander said he still prefers it the other way around, so keeping the read/write copy in-house even though the performance might be better in the Cloud.

Sander objected to the choice of cloud providers. All of the initially chosen providers are US companies. He said it will force them into disclosing data and believes cloud providers should be from our own region, also to support local companies. He asked why that is the case. Kaveh responded that they struggled to find a provider in the EU that fulfilled their criteria but are definitely open to input. On the legal side, this becomes more important when having to migrate critical data, and they will make sure the data is hosted in European data centres. Sander said that Microsoft, even though their data centre is in Europe, they are still a US company. Kaveh said that for now all data is publicly available but that Sander has a point.

Anna Wilson, HEAnet, had doubts about multi-cloud, which stems from having to maintain two clouds, requiring a wider skillset which is not easy to get. She suggested thinking about the future as well. Decisions that are made now will affect ticket times and service outages later, so make sure that staff is skilled by deploying this in a cautious way and stay transparent about it. Kaveh responded that the RIPE NCC is on the same page and wants to provide long-term services together with the membership. Razvan added that it indeed does not make sense to duplicate everything. They will only do it for critical applications and then decide on a case-by-case basis.

Peter Hessler, DENIC, asked how any US sanctions would affect members of our service region that are affected by them. Kaveh responded that from a contractual level, members are not affected. But if it happens, they have proper measures in place, like if a service is blocked. Ultimately, the RIPE NCC is in full control of the service so they can migrate it back to in-house. Razvan suggested catching up with Peter afterwards for a virtual chat to further talk about this.

Markus Zeilinger, University of Applied Sciences Upper Austria, said that the benefit of the cloud is that services can scale up/down in (nearly) real-time. He asked if this is necessary/critical for RPKI and the RIPE Database. Kaveh responded that there is no big need for that at the moment for these two services, but that this is a good possibility especially for larger datasets, like RIS and RIPEstat data.

Jan Žorž, 6connect, asked if we expect to start downloading/updating RIPE trust anchors TAL from AWS or Google cloud in the future. Kaveh responded that it is not in the plans. However, he said that what they will learn from hosting our keys in the cloud can be very useful for the future where people will hopefully sign with their own key at their own place and then publish.

Erik Bais, A2B Internet, stated that no IPv6 deployment should be a “go-no-go” requirement. Kaveh responded that Razvan wanted it listed in the criteria and it is. Razvan added later on that everything they have migrated so far is IPv6 reachable. Erik asked if there is a hard cap in the invoicing for the charges of the cloud provider once the services are in the cloud? Kaveh responded that they have hard caps in and they are fully in control. Erik asked if there is any requirement on whether the cloud provider is actually going to be present in the next 10 years with the service. Kaveh said this was one of the criteria and it was looked into. They looked into the general health history and all, although nobody can predict the future. Razvan added that history serves as an indicator for the future.

Dmitry Kohmanyuk, Hostmaster.UA, asked if he read it correctly that “two clouds” are considered an option, with “one cloud and one on premises”. He said that with the big A and big G, it looks like no data centre is needed at all. Kaveh responded that they are looking at a multitude of services, not just WHOIS/RPKI, but also our internal systems, financial system, and knowledge management system. Therefore, it would really depend on the service. They want to work on this together with the community and there is no plan to have no data centre.

Daniel Karrenberg, RIPE NCC, made a comment that he sees people get passionate about the topic and recommends everyone frame this topic in terms of requirements rather than solutions. We should think of what the community thinks are the requirements on the core services of the RIPE NCC, and linked to the upcoming GM, also the costs that these requirements bring. Kaveh agreed that we should think about long term operations of the RIPE NCC, how we can provide the best and most reliable service that is also cost effective, but that it is not about money. It is about being sustainable as an organisation. He also mentioned that the cloud was not something coming from management to staff but rather bottom up, coming from the engineers. It is important to keep engaging and get feedback and input.

E. NCC Services Chair Selection

The presentation is available at:
https://ripe82.ripe.net/archives/video/586/

F. Open Microphone Session 

The presentation is available at:
https://ripe82.ripe.net/archives/video/587/

 Z. Any other business

None and no presentation available.

End of session.