Clean-up of Unreferenced Data

Introduction

Any data entered into the RIPE Database must comply with the RIPE Database Terms and Conditions. Over a period of time, data is changed and references are removed. This often results in many of the secondary objects (for example, person, role and mntner objects) being left in the Database and serving no useful purpose. In order to keep the Database clean and to comply with data protection rules, this unnecessary data must be removed. New personal data should not be stored in the RIPE Database beyond a reasonable "work in progress" period before it is referenced by operational data objects.

Authority

Cleaning up this data has been discussed many times at RIPE Meetings and on the mailing list of the RIPE Database Working Group.

The authority to clean-up this data was first given to the RIPE NCC at the RIPE 40 Meeting in Prague in 2001

The scope was expanded at the RIPE 41 Meeting in Amsterdam in 2002.

A clean-up process to do periodic clean-ups was put in place in 2003. However, this process proved to be inadequate and the number of unreferenced objects continued to grow. The current numbers are shown in a graph produced by dbconstat.

The issue was raised again at the RIPE 54 Meeting in Tallinn in 2007, continued on the RIPE Database Working Group mailing list throughout 2007 and then consensus was reached at the RIPE 56 Meeting in Berlin in 2008.

White Pages

The RIPE community recognised that some well-known people within the Internet industry do not have references to their person objects. It is beneficial to the community for those people to maintain a presence in the RIPE Database through their person objects. In order to allow their person objects to exist without being deleted by this clean-up process, the RIPE NCC implemented a White Pages facility. Check out the details if you believe this applies to you.

Process

The object types to be deleted in this clean-up process are:

• person

• role

• mntner

We currently use a simplistic model. If the person or role object is unreferenced, it is eligible to be deleted. If a person or role object is only referenced in a mntner object and the mntner object only references that person or role object, then both the mntner and the person or role objects are eligible to be deleted.

To be deleted, they must be unreferenced (or mutually referencing) for a set number of consecutive days. This is currently set at 21 days.

Notifications of the deletion of objects will be sent to anyone who has optionally included a "notify:" attribute in the person, role or mntner objects, or a "mnt-nfy:" attribute in the mntner objects.

Implementation

The script is currently producing statistics on how many objects are eligible for deletion. It will start deleting these objects on 26 January 2009. Initially, only a small number of the eligible objects will be deleted each day. This will allow RIPE NCC Customer Services to assess the impact of the clean-up process. Based on their feedback, the number of objects deleted will be increased in order to clear the backlog of unreferenced objects that has built up.

Once this backlog has been removed, all newly unreferenced objects will be eligible for deletion after the set number of consecutive days.