[db-wg] Complying with Mail Provider Requirements When Sending Mail from Whois

Hi Denis,

> On 21 Mar 2024, at 04:03, denis walker via db-wg <db-wg at ripe.net> wrote:
> 
> Hi Sander and others
> 
> On Wed, 20 Mar 2024 at 23:06, Sander Steffann <sander at steffann.nl> wrote:
>> 
>> Hi Denis,
>> 
>>> So we just stay where we are, yet again, with the NCC sending out tens
>>> of thousands of emails every day to some of the almost a million email
>>> addresses in the database and you all continue to get spam sent to
>>> those harvested email addresses.
>> 
>> You seem to perceive a problem that not everybody else is perceiving. Maybe we should first see what the users of the database think of this instead of assuming there is a problem.
> 
> Not everybody has OCD.
> 
> I know this is a long email. I know few people will read it. I don't
> expect anyone to respond to it as I talk about issues that no one else
> in this community will talk about, no matter how important they are.
> But as I say in my disclaimer, my mind is driven by detail. Things I
> see need to be said and archived, even if nothing is done.
> 
> I would LOVE to hear what users of the database think. A 'perceived'
> problem, HHmmm.
> 
> -The database design and data model are about 30 years old
> -the current java based software was first written about 12 years ago
> -the data model was built around storing unedited, partially
> un-formatted, human readable text blocks, with all indexed and machine
> parsable data a fudge on top of the text blocks
> -few centralised, distributed tools for processing data from the
> database, it's mostly diy
> -everytime someone asks for a new feature, like assigning an
> allocation, it needs open heart surgery
> -2023-04 has highlighted that the community has collectively forgotten
> what some of the data means or why rules were put in place
> -2023-04 has also shown that we have lost track of what a public
> registry is, who it serves and in what way
> -we have no documented business requirements for either the RIPE
> Database or a public registry
> -there is still a mindset problem with many users still seeing it as
> an operators tool and denying the need to serve other stakeholders
> -it has security issues
> -it has privacy issues
> -it has commercial confidentiality issues
> -geofeed references to external data (itself not covered by policy)
> allows any policy constraints on the database to be undermined
> -there is no simple way for data users to even query their own data,
> never mind manage it
> -2023-04 has also stressed that it is a very manual process to update
> the database which is not convenient for operators
> -it is not possible to have a full audit trail of all changes to your
> own data, despite the notif type attributes
> -the data model does not handle different business models adopted by
> some resource holders now, especially where resource holders are
> financial investors and some brokers operate as commercial IRs below
> the RIPE NCC
> -it operates in an evolving environment increasingly subject to
> regulatory control that is outside the control of this community,
> while the database itself remains static
> -no possibility of introducing new concepts like customised
> authentication or verified query users
> -the lack of simple concepts like inheritance results in massive
> duplication of data across millions of objects
> ....
> 

I'd like to make a clarification, in that any functional issues with the RIPE database should be kept separate from the technical implementation, which is the responsibility of the Database team.

The current Java based software was written to make the database more maintainable, including adding new features, and this has been done very successfully over the past 12 years.

The DB team is productive with the current software, i.e. we deliver features in a reasonable amount of time with a good quality level. Our code is open source so can be inspected and re-used. The other Software Engineering teams also use Java, so we can cooperate easily.

We keep Java and MariaDB updated to the latest LTS (Long Term Support) versions so we can make use of latest features combined with a stable release.

Operationally we deliver excellent uptime of the Whois service. We are very familiar with how the software behaves in production, in particular monitoring and handling failures.

There may be *functional* issues with the RIPE database, but if the community agrees on changes to the data model, we are confident that we can deliver any resulting *technical* changes with the software we already have.

Regards
Ed Shryane
RIPE NCC