[db-wg] Proposal - Maintaining person, role and domain objects

[Apologies for duplicate mails]

Dear Colleagues,

As a result of discussions during the Database Working Group session at 
RIPE 54, the RIPE NCC has nine proposals and implementation plans to 
present to the community. Although most of them are now in the final 
stages of preparation, we will send them out one at a time over the next 
few weeks for consideration by the community.

The first one concerns maintaining all objects in the RIPE Database, which
followed from a recommendation from the Data Protection Task Force (DP TF)
(see below). We have already had some preliminary discussions about this
with the DP TF. They provided the RIPE NCC with some very useful feedback,
which is incorporated in this proposal.

Regards
Denis Walker
RIPE NCC



Maintaining person, role and domain objects
-------------------------------------------
 
  * Introduction 
  * Proposal 
  * Illustrations of how the checks work 
    - New startup 
    - Closing down 
    - Modifying existing data 
  * Implementation 
    - Stage 1 
    - Stage 2 
    - Stage 3 
  * Statistics
 
 
Introduction
------------
It was agreed at RIPE 54 that the "mnt-by:" attribute would be made
mandatory on all objects. Currently, for person, role and domain
objects, "mnt-by:" is optional. Making "mnt-by:" mandatory on person
objects raises two clear issues: 
 
  1. It creates a chicken and egg situation between person and mntner
     objects. 
  2. Many person objects are not changed for years, so any change to
     syntax may take a long time to be reflected in the database. Many
     un-maintained person objects could therefore remain so for many
     years. 

Proposal
--------
Making "mnt-by:" mandatory on role and domain objects does not cause
any technical difficulties.
 
To address all the issues relating to the person objects, we suggest a
slightly different approach.
 
  1. Make "mnt-by:" mandatory in person objects. 
  2. Provide a mechanism in dbupdate to handle the startup case where a
     new person and mntner objects are required. 
  3. Provide a mechanism in dbupdate to handle the closing down
     situation where the last person and mntner objects are to be
     deleted. 
  4. Extend the referential integrity checks so that a person object
     cannot be referenced unless it is maintained, except in a mntner
     object.
  5. Extend the referential integrity checks so that a mntner object
     cannot be referenced unless the person objects it references are
     maintained. This does not include the cyclic case of a person and
     mntner object referencing each other.
 
The referencing checks sound complicated in words, but are actually
quite simple. The illustrations below explain how it works in practice.
 
This approach has a number of advantages:
 
  1. It accommodates the chicken and egg situation. 
  2. We already do referential integrity checks. Whenever there is a
     reference to a person object, the database software checks if the
     person object exists. We only have to extend this check to see if
     the person object is also maintained. 
  3. Whenever there is a reference to a mntner object, the database
     software checks if the mntner object exists. We only have to extend
     this check to see if the person objects referenced by the mntner
     are maintained. 
  4. Each time any other object (say an inetnum or aut-num) is
     modified, the referenced person objects need to be maintained. If
     not, the update will fail. This will not prevent anyone from working.
     If they are going to modify an inetnum object, they must be authorised
     to do so. They can apply the same mntner to any un-maintained person
     objects. This encourages users to maintain the existing person
     objects. A CGI script may be provided to make this process simple and
     quick if it is considered to be helpful.
  5. No un-maintained person object can be linked to the white pages to
     avoid deletion. (The white pages will be explained in a later proposal.)
 
 
Illustrations of how the checks work
------------------------------------
 
New startup 
Send an update message to dbupdate to create a person object and a
mntner object. These must be the first two objects in an update
message, in any order. The references to each other must also be in
place. The database software will accommodate this. 
 
  person:         Den is
  address:        RIPE Network Coordination Centre (NCC)
  address:        Singel 258
  address:        1016 AB Amsterdam
  address:        The Netherlands
  phone:          +31 20 535 4444
  nic-hdl:        DW-RIPE
  mnt-by:         aardvark-mnt
  notify:         denis at ripe.net
  changed:        denis at ripe.net 20040318
  source:         RIPE
 
  mntner:         AARDVARK-MNT
  descr:          Mntner for denis' objects.
  admin-c:        DW-RIPE
  tech-c:         DW-RIPE
  upd-to:         denis at ripe.net
  auth:           X509-1
  notify:         denis at ripe.net
  mnt-by:         AARDVARK-MNT
  referral-by:    RIPE-DBM-MNT
  changed:        denis at ripe.net 20040225
  source:         RIPE
 
The update software will recognise that the first person/mntner object
references a non-existent mntner/person object. It will check that this
referenced object is next in the update message. If it is, the "mnt-
by:" attribute will be removed from the person object. The person and
mntner objects will be created, and then the person object will be
modified to add back the "mnt-by:" attribute.
 
The objects are now fully configured and can be used. The person object
can be referenced by any other object where a nic-hdl is referenced. It
can also be linked to the white pages. The mntner can be used to
protect any data in the database.
 
If the non-existent referenced object is not next in the update message
then an error message will be generated and the update will fail. This
is the current behaviour.


Closing down
The same situation occurs when closing down and deleting all the data.
It is possible to delete all data except for the last person and mntner
objects in the normal way. To delete these last two, send them together
in an update message, both marked with the "delete:" pseudo attribute.
Only these two objects should be in the update message.
 
The update software will recognise that the first person/mntner object
to delete is referenced in a mntner/person object. It will check that
this referencing object is next in the update message and also marked
for deletion. If so the person object will be modified first to remove
the "mnt-by:" attribute. The mntner object will then be deleted,
followed by the person object.


Modifying existing data 
Suppose some data is to be modified and there is a reference to an un-
maintained person object.
 
Suppose you already have this data in the database: 
 
  inetnum:        193.0.0.0 - 193.0.7.255
  netname:        RIPE-NCC
  descr:          RIPE Network Coordination Centre
  descr:          Amsterdam, Netherlands
  remarks:        Used for RIPE NCC infrastructure.
  country:        NL
  admin-c:        DW-RIPE
  tech-c:         DW-RIPE
  status:         ASSIGNED PI
  mnt-by:         AARDVARK-MNT
  mnt-lower:      AARDVARK-MNT
  changed:        bit-bucket at ripe.net 20060221
  source:         RIPE
 
  person:         Den is
  address:        RIPE Network Coordination Centre (NCC)
  address:        Singel 258
  address:        1016 AB Amsterdam
  address:        The Netherlands
  phone:          +31 20 535 4444
  nic-hdl:        DW-RIPE
  notify:         denis at ripe.net
  changed:        denis at ripe.net 20040318
  source:         RIPE
 
  mntner:         AARDVARK-MNT
  descr:          Mntner for denis' objects.
  admin-c:        DW-RIPE
  tech-c:         DW-RIPE
  upd-to:         denis at ripe.net
  auth:           X509-1
  notify:         denis at ripe.net
  mnt-by:         AARDVARK-MNT
  referral-by:    RIPE-DBM-MNT
  changed:        denis at ripe.net 20040225
  source:         RIPE
 
The inetnum and mntner objects both reference a person object that is
not maintained. This situation can continue for the foreseeable future
with no problem. The situation becomes problematic when some data needs
to be changed.
 
An attempt to modify the inetnum object will fail. This is because it
references a person object that is not maintained. It also references a
mntner object which references un-maintained person objects. (This will
only happen in stage three of the implementation)
 
These person objects will have to be maintained before the inetnum
object can be modified. A CGI script will be provided to allow this to
be done quickly and easily.
 
When all referenced person objects are maintained, the original update
can be resubmitted and will proceed.
 

Implementation
As with the CRYPT-PW deprecation this will have a staged rollout.
 
Stage 1
 
  * No new person, role or domain objects can be created without a
    "mnt-by:" attribute. 
  * Any un-maintained person, role or domain object cannot be modified
    without adding a "mnt-by:" attribute. 
  * Any update where objects reference an un-maintained person object,
    either directly or through a mntner with such references, will
    generate a warning message in the acknowledgement.
 
In this stage the acknowledgement message may include these warnings:
 
  ***WARNING: Un-maintained person object referenced [DW-RIPE]
  ***WARNING: Un-maintained person object referenced [DW-RIPE] in
  mntner [AARDVARK-MNT]
 
Stage 2
 
  * Any update where objects reference an un-maintained person object,
    either directly or through a mntner with such references, will
    generate a warning message in the acknowledgement. 
  * Any NEW reference to an un-maintained person object or to a mntner
    which has such references will generate an error message in the
    acknowledgement and the update will fail.
 
In this stage the acknowledgement message may include these warnings
and errors:
 
  ***WARNING: Un-maintained person object referenced [DW-RIPE]
  ***WARNING: Un-maintained person object referenced [DW-RIPE] in
  mntner [AARDVARK-MNT]
  ***ERROR: New reference to un-maintained person object [DW-RIPE]
  ***ERROR: New reference to un-maintained person object [DW-RIPE] in
  mntner [AARDVARK-MNT] 
 
Stage 3

  * Any update where objects reference an un-maintained person object,
    either directly or through a mntner with such references, will
    generate an error message in the acknowledgement and the update
    will fail.
 
In this stage the acknowledgement message may include these errors:
 
  ***ERROR: Un-maintained person object referenced [DW-RIPE]
  ***ERROR: Un-maintained person object referenced [DW-RIPE] in mntner
  [AARDVARK-MNT] 
 
 
Statistics
----------
While not a very statistically valid survey, we looked at a few days
just after the RIPE meeting to see how many new person objects were
created with and without mntner objects. We also queried the new
objects some time after creation to allow for a "mnt-by:" to be added
later. We also noted how many unique person objects were referenced in
any objects in update messages with and without mntner objects. AUTO-
references were ignored in both creations and updates, and multiple
instances of the same person object being referenced many times were
counted as one.
 
   --------------------------------------------------------
  |   Date   |Created  |Created  |Referenced  |Referenced  |
  |          |with     |without  |with        |without     |
  |--------------------------------------------------------|
  | 20070515 |  156    |   89    |    925     |    726     |
  | 20070516 |  111    |   67    |   1022     |    486     |
  | 20070517 |   85    |   48    |    476     |    185     |
  | 20070518 |   82    |   18    |    679     |    445     |
   --------------------------------------------------------