Re: [db-wg] restoring person object

[Anton Elita aelita@localhost]

Denis,

thanks for the extensive explanation, much appreciated!
I do not see however the answer to: "why there is no prevention before actual deletion", followed by "why there is no grace time"

to my understanding both will not need additional human resources from RIPE and will save most of the "good" records without a reference from being deleted forever.

best regards,
 anton

On Tue, Sep 22, 2009 at 4:24 PM, Denis Walker <

> wrote:

Dear Colleagues,

Apologies in advance for the length of this email, but many issues have
been touched on in this thread. I feel these need to be explained with
some background information so that members will understand why we
implemented these features. If the community wishes to change this
implementation, within the constraints of legal requirements, we will of
course make any necessary changes or adjustments.

- Why delete objects?

Since 2006, the RIPE Data Protection Task Force (DP TF) has been meeting
 to consider many aspects of the RIPE Database from a legal and privacy
point of view. This task force is made up of mainly RIPE NCC members
with technical input from the RIPE NCC and legal input from lawyers.

This DP TF has established Terms and Conditions (T&C) for the RIPE
Database. These T&C define the purpose of the RIPE Database and the
basis for storing information in it. Personal data is of particular
concern. We are only allowed to store personal data if it supports the
primary objects in the Database, such as the INETNUM and AUT-NUM. The
data protection laws do not allow us to keep personal data indefinitely.
In addition, they only allow us to keep personal data if it matches the
defined purposes of the Database as listed in the T&C. There is no legal
basis for us to store personal data used for other services external to
the RIPE Database.

It was agreed by the DP TF that PERSON and ROLE objects that remain
unreferenced by any primary objects for a period of time would be
automatically deleted. This includes, for example, a PERSON object
referenced in a ROLE object, referenced in a MNTNER object that only
maintains these PERSON and ROLE objects. If these objects form a
self-contained set, unreferenced outside of this set, then they will be
deleted.

Currently, the period of time before automatic deletion is set at three
weeks. This can be adjusted if it is not considered long enough. In
order to avoid excessive emails, we do not send a notification before an
object is automatically deleted. Notifications are sent when the objects
 have been deleted. If users are made aware of the process they will
know when they remove all references to a PERSON or ROLE object it is
liable for deletion if left in that state beyond the defined work in
progress period.

- Why did we suspend the cleanup?

At the DP TF meeting at RIPE 58, we learned something new about the
daily operations of some of our members. We discovered that some members
maintain their own internal database with a copy of their information
that is contained in the RIPE Database. They submit updates to the RIPE
Database based on the data contained in their own database. This depends
on the two databases remaining synchronised.

However, our garbage clean-up process was deleting PERSON and ROLE
objects. The NIC handles released by these deletions were immediately
available for re-use. Any NIC handle with a low number for a particular
letter combination was highly likely to be re-used quickly by anyone
else creating objects with an AUTO-NIC handle. If the deleted PERSON
object has no "notify:" attribute and is either not maintained or the
maintainer has no "mnt-nfy:" attribute, then no notification will be
sent when it is deleted.

This means there could easily be situations where a PERSON object is
deleted and the member is not aware of this. In such a scenario, a new
PERSON object could be created with the same NIC handle. The member
would reference what he believes to be his PERSON object based on the
information in his own database, but in fact a completely different
person would be referenced. As no authorisation is required to reference
a PERSON object and no notification is sent when such a reference is
made, no one will know a mistake has been made.

As this is a serious privacy issue, we suspended the clean-up process
until we had a solution.

- Why do we no longer allow re-use of NIC handles?

This is also an issue that was considered by the DP TF. Dutch case law
considers a NIC handle to be personal data. By allowing NIC handles to
be re-used we are allowing multiple people to be identified by the same
'personal identifier'. From a legal perspective, this is unacceptable.
This personal aspect of NIC handles is also why we will soon deploy new
code to remove NIC handles from Near Real Time Monitoring (NRTM) data
streams and the daily object split files.

In addition to legal issues, ensuring NIC handles remain unique over
time simplifies many other matters. For example, it solves the problem
concerning the clean-up process. Once a PERSON object is deleted, that
NIC handle will not be re-created again. So if a member tries to
reference it, not realising it has been deleted, the update will fail
rather than possibly referencing the wrong person.

Some people like to create personalised NIC handles. As with the POEM
object, it is fun to have these things, but there is no operational
basis for personalised NIC handles. It is simply a reference into the
Database. If you can find one that has not been used before you may
consider that a bonus. But if it is not referenced, it will soon be
deleted and cannot be re-created.

- Why did we restart the clean-up process?

Deploying the feature that prevented re-use of NIC handles removed the
issue that caused us to suspend the clean-up process. We re-started this
process on 12 September 2009.

- Why don't we offer a service to re-create accidentally deleted objects?

There are many problems with doing this. In an ideal situation, where
the PERSON object was maintained and a user can provide the
authentication, we could re-create the object as it was when deleted.
But our Customer Services staff would have to process the initial
request, recover the old object, validate the authentication and then
re-create the object with an override option. This would have to be done
in the same way that we now use for recovery of a lost password or key
for a maintainer.

In a situation where a PERSON object was not maintained, did not have
any notify email addresses and was created by a different person on
behalf of a customer, it would be very difficult to find a procedure to
validate such a request.

If we provide this service, we might have to put a lot of time and
resources into recovering 'lost' NIC handles. However, if the community
wants this, it can be done.

- What is the White Pages feature for?

This was set up for a very specific reason. A small number of people who
have a significant presence in the community, but who do not directly
manage operational data, can "opt in" to having their personal data
stored in the RIPE Database. This is not so that their NIC handle can be
used in external services. It is to make their contact details available
to the community through the RIPE Database. From a legal point of view,
personal data must be proportionate to the purpose for storing it. As
long as the White Pages feature is low volume, it satisfies this
requirement. It is not intended to add large numbers of people simply to
protect their NIC handles in case they ever become un-referenced from
the operational data they relate to now.

- Multiple feature implementations

One of the problems we are currently encountering is the implementation
of multiple features in the same area of the code. There are many legal,
privacy and technical issues concerning personal data. These are
starting to overlap and have side effects. There are still issues that
have not been addressed yet. One of these is maintaining personal data.
Before RIPE 59, the RIPE NCC will set up a new test server to allow RIPE
 Database users to check their internal processes in an environment
where all objects must be maintained.

The original RIPE Database design did not take into account privacy
issues. The RIPE NCC will continue to work with the DP TF and the
community to find acceptable solutions to these issues.

- RIPE 59

These issues will be discussed at RIPE 59. I hope I have explained
enough of the background to allow the discussion to move forward.

Regards
Denis Walker
Business Analyst Database Group
RIPE NCC