[db-wg] "Hidden" historical WHOIS records

Ronald,

On 30/01/2021 05.42, Ronald F. Guilmette via db-wg wrote:
> 
>     https://www.ripe.net/ripe/mail/archives/db-wg/2020-December/006736.html
>

When you refer to the two question that were ignored, I guess you mean:

 > >True...historical versions of these objects are not available in any
 > >form for privacy and security reasons.
 >
 > Ok, so, two questions:
 >
 > 1)  Is that based on community policy, or on internal RIPE NCC policy?
 >
 > 2)  What are these "privacy and security reasons", exactly?

Keep in mind that my participation in this was between 20 and 15 years 
ago, and my memory is likely imperfect and I invite other people 
involved to clarify or correct if they desire.

As far as I know the ability to keep historical data was introduced with 
the implementation of the database deployed in 2000. I started working 
at the RIPE NCC a few months before the deployment and helped with the 
it, although I was not in a decision-making position at the time. Lack 
of historical data before that was not a matter of policy, but a 
limitation of the technology; the database before that did not keep 
older versions.

During this era many ccTLD within the RIPE region (which at that time 
included Europe, the Middle East, and northern Africa) used the RIPE 
Database either as a primary or backup record of their domain 
delegations. This was a service provided by the RIPE NCC to help 
encourage the growth of the Internet within its service region; I 
believe that this fits in well with the stated purpose of the RIPE NCC, 
and was partially responsible for the success of ccTLD.

The ccTLD data started to become a problem as various ccTLD became 
wildly popular, and domain objects and the associated objects (person, 
role, maintainer) consumed a large amount of storage, compute, and 
network resources. The large amount of personal data also highlighted 
the privacy problems associated with keeping such contact data, as there 
were lots of private information stored and published.

When I mention "privacy problems", in 2000 the biggest concern of people 
was spam. Anti-spam techniques (both technical and legal) were in early 
days and the spammers were basically winning the arms race. There were 
no social platforms so e-mail was too important to ignore, but also 
becoming increasingly flooded. Given this, people were understandably 
very sensitive to having their e-mail address published.

Given these issues, I do not think we ever considered providing access 
to historical data about person or role objects. I do not think it was 
removed, rather never provided.


There were other ways that database data was available.

The RIPE NCC was already providing daily dumps of the database on the 
FTP site for years before this, but I'm pretty sure the person and role 
objects were not available for download due to privacy concerns. At one 
point the dumps of other data types were modified to filter out 
references to person & role objects, to prevent people from getting a 
complete list of "active" persons or roles that they could then query. 
Additional filtering was added in the early 2000's such as removing 
CRYPT-PW and MD5-PW attributes to prevent dictionary attacks against 
maintainer passwords.

The RIPE NCC also provided NRTM (near-real-time mirrors), which provided 
full information, although getting access to such a feed required a 
contract limiting how the information could be used or distributed. I 
believe such a full feed was eventually removed or more severely 
restricted as well.


The RIPE NCC during that era worked closely with the community, and 
always presented such changes to the community beforehand and reported 
when they were complete. There was generally a presumption of good 
faith, which I think was deserved and I think the RIPE NCC still 
deserves. I'm not going to search through decades-old mailing list 
archives or review PowerPoint presentations from that time to garner the 
details; I leave that to historians, or to folks interested in 
litigating decisions made long ago.

Cheers,

--
Shane