FYI: new database code in production

Dear all,

Hereby, I want to announce that we have put the new RIPE database release
2.00 in production. Since this version is a major upgrade, we certainly
expect a few problems (unfortunately we already found some and fixed
them already ;-)). Please take a close look at your updates and send me
(<ripe-dbm at ripe.net> only) your bug reports so that I can fix any
possible problems.

I am still finalizing the documentation (with more detailed information
on the new features) and the software distribution package. You can
expect a separate announcement from me on this later this week.

And finally:

I want to apologize for the late announcement of this upgrade and the
confusion that might have been caused by this.

If you have any further questions, please don't hesitate to contact me,

Kind regards,

David K.
-----

*****Preliminary release notes*****:


Dear all,

This message is to announce the new 2.00 release version of the RIPE
database. Many new features and bug fixes have been added since the the
previous beta release.

The following lists the most interesting new features since the previous
non-beta release:

- AUTO NIC handle assignment support 

- hierarchical authorization for creation of 'inetnum' and 'domain'
  objects.

- role object support

- prefix notation for 'inetnum:' updates is supported and IP range
  notation for whois lookups

- extended 'country:' attribute syntax implemented, more then one coutry
  on a line and more 'country:' attributes per object are allowed now.

- the special RIPE whois client is now fully supported and included in
  the distribution.

- much stronger syntax checking (=as strong as specified by the original
  database specification) for NIC handles, phone numbers, E-mail
  addresses, domain names (Some people might not like this ;-)).

- network updates, a method for directly updating objects by use of the
  program 'networkupdate'. The use of this method is restricted by 
  an accesslist and intended for the maintenance people of the database.
  
- BSD DB dbm package is fully supported. This dbm package is a fast,
  platform independant and reliable dbm package.

- No limitations on the number of hits (no matter what indexing package
  you use): 
  a query like 'David' will give you an answer and person names like
  'John George' will be found
  
- new and more intelligent indexing program 'indexdb' that can do a raw
  index or a clean index and automatically detects if it is dealing with
  a classless index. netdbm & cleandb are now obsoleted.

- reverse lookup of objects (whois -i option), this enables you to find
  for example the routes with a given origin as by doing:

  whois -h whois.ripe.net -i origin AS3333

- Nearly real time mirroring of the RIPE database. This feature is fully
  supported but has a few shortcomings that will involve a bit more
  administration work until the 'stored/processed' attribute and better
  crash recovery routines are implemented.

- experimental inet6num support for ipv6 network objects, including
  support for -L, -m, -M queries.

and many more small changes and fixes. Please read the release notes at
the bottom of this message for more information and discussion about
these topics.

Please send bug reports/fixes directly to <ripe-dbm at ripe.net> and I will
fix the problem but be fast since I will move to another job at ISI in
August!

Thanks for all your help,

Kind regards,

David Kessens
RIPE NCC software maintainer
-------


RELEASE NOTES (RIPE database version 2.00):


- Nearly real time mirroring support:
  
  What is it:
  
  The database software now keeps serial numbers of all updates. If you
  have been added to an accesslist, it it possible to retrieve all the
  changes to the database. A program 'syncdb' has been added to the
  distribution that can retrieve these changes and update your mirror
  database automatically. This program is intended to be put in a
  cronjob.
  
  The whois erver has an access list so you first need to ask the site
  to be mirrored to add your host to the access list.

  Please let me now at <ripe-dbm at ripe.net> if you are interested in
  trying out this feature, and I will add the IP number of your site to
  the access list.
  
  Short discussion:
  
  Note that this is the first version, it currently uses seperate files
  internally for every update. We plan to change this since this is not
  really what we want with 2000 (atomic) updates on some days. 
  
  Furthermore, we use currently an ever increasing integer serial number.
  A proposal (stored/processed attribute) on how to this right has
  already been circulated to the RIPE database working group and discussed
  on the RIPE meeting. Implementation of this attribute will also allow
  for a better crash recovery behavior.


- No limitations on the number of hits (no matter what indexing package
  you use): a query like 'David' will give you an answer and person
  names like 'John George' will be found

  
- New and more intelligent indexing program 'indexdb' that can do a raw
  index or a clean index and automatically detects if it is dealing with
  a classless index (from the config file). netdbm & cleandb are now
  obsoleted. The indexing speed has also been improved.

  
- BSD berkeley DB support:

  The database software can now use the Berkeley DB dbm package.
  According to many people this package is superior to most other dbm
  packages that perl can use (unlimited size of keys/values, platform
  independent db files). We have put the current sources for this
  package at ftp://ftp.ripe.net/tools/db.1.85.tar.gz

  It should be possible with the BSD DB package to increase the
  OVERFLOWSIZE and as result get rid of (most) of the overflow files.
  However, it is not clear if this is desirable since it doesn't seem to
  make the software more memory/space/speed efficient.


- a new help and HOWTO use the RIPE database file is available
  
  use 'whois -h whois.ripe.net HELP' to get the most recent version.
  
  The document gives a short introduction on the database, tells you how
  to create/update/delete objects, contains a section with tips and common
  problems and gives pointers to other documentation about the database.
  
  
- BSDI/perl memory leak problems

  Due to a memory leak with perl on BSDI machines we needed to make the
  code more memory conscious. enread.pl has been rewritten completely and
  uses now less memory and as an (intended) side effect became faster
  when reading big objects. Also the special input routine for reading in
  an object from an E-mail has undergone the same optimizations.


- Most patches (for greater speed and less memory consumption) that I
  received from Curtis Villamizar <curtis at ans.net> have been included.
  Thanks Curtis!
  
      
- The direct network updating method is now fully supported.

  Currently this method is only intended for use by the registry itself,
  however it is possible to add support in the maintainer objects if
  desired by the RIPE community.
  
  You can now do you updates very rapidly by giving the command
  'networkupdate' and input your object via STDIN. Of course, this is
  also protected with an access list.


- The special magic to overrule the maintainer protection needs a
  password now. Syntax will be published after the main registries
  support this...


- All support for the historical 'guarded' objects have been removed. The
  guardian attribute can be obsoleted if the database working group
  wishes to. Present guardian attributes will act as a 'notify:'
  attribute.


- the special RIPE whois version is now fully supported and included in
  the distribution.


- cryptpw

  is a new (small) program that encrypts a password with a shoosen salt
  for use in the maintainer objects.
  
- dbupdate -T
  
  Makes it very easy to install your own test database. The only
  difference with a normal update is that it doesn't need manual
  authorization for creation of maintainer objects and that it will add a
  warning to your acknowlegment message that this is not the normal
  behavior...


- status: attribute as described in ripe-127 fully supported


- whoisd changes:

  -D runs in daemon mode otherwise in 'inetd' mode ('inetd' mode is
     untested)
  
  The number of disk accesses have been further minimized and should
  result in a better performance.

    
- line continuation by use of a "\" character at the end of an attribute
  line in your update message. The database software will then
  automatically concatenate the next line with the previous one.


- prefix notation for 'inetnum:' updates and IP range notation for whois
  lookups is supported


- extended 'country:' attribute syntax implemented. You can now specify
  multiple countries in a 'country:' attribute line.


- much stronger syntax checking for NIC handles, phone numbers, E-mail
  addresses, domain names (Some people might not like this ;-)). The
  syntax was too relaxed and follows our own specifications more closely.


- perl5 patches. perl5 is still not supported. However, the biggest
  problems should be gone now (the software runs with perl5 at home with
  Linux & perl5.001m). The Makefiles have been changed to support
  perl5 and some of the database tools as 'whois' can be configured to
  run with perl5 by default (and work with it!). See the Makefiles on how
  to configure this. You are welcome to experiment with perl5 and we are
  interested in your comments!

  
- Tony Bates <Tony.Bates at mci.net> recent patch for a problem with the
  maintainer notifications. Thank you Tony. 


- databases and serial number files that don't exist are now
  autogenerated.


- Config file has been simplified by removing some options that can be
  determined by the database software itself. Of course, also a few new
  options appeared. Everything is documented in the config file itself.


- The Makefiles have been rewritten to allow for easier installation and
  more logical behavior when doing make [[un]install].

  
- many small fixes and clean up of the code to make above changes
  possible. 
  
  - bug that caused problems when deleting objects with(out) syntactic sugar

  - bug that caused a crash when one tries to delete an object
    that cannot be updated (object is mirrored from other database)


- List of OS's that we know the RIPE database works with:
  
  - SUNOS
  - BSDI
  - Linux
  
  (Please inform me with you own results so that I can update this list!)
  
  Automatic on the fly patches implemented in the new Makefiles should
  make it possible to run the database on Solaris. This is untested for
  now and you will need a special perl4 that will be made available from
  ftp://ftp.ripe.net/tools. Peter Haag has made the perl version and
  patches available to me. Thanks!
 
  
- The auto nic handle generation support (Gabor Kiss helped me with some
  patches)


- Role object


What didn't make it in this release:

- checking for relations between objects:

  - disallow deletion of referenced object
  - disallow creation of objects that reference non exiting objects