Databases Synchronization Proposal

On Thu, 2 Mar 1995 10:23:27 -0500 (EST)  Laurent Joncheray wrote:
> 	Within the proposed model the incoming Processing entity will
> forward a copy of the mail to the registry B's incoming Processing
> entity.  In order to prevent a new report to be sent to the user by the
> registry B's process, the sender of the mail will be set to a special
> generic address ("dbadmin at Registry_A") which will be recognized by the
> registry B and won't generate a report to the sender.
[Left out the rest of the proposal for brevity]

Laurent,

I have been thinking on similar lines too. There is one catch, which you
might want to include in your proposal.

There is semantic information in the order in which updates are processed.
Assume the following scenario:
1. Someone registers a Bananna object (note typo)
2. The database includes & sends ack
3. The person finds the typo
4. The person deletes Bananna
5. The person registers Banana (no typo).

If these updates are also sent to a shadow copy, then one must make
shure that (1) comes before (4), otherwise the delete won't work
and Bananna will still be in (the databases will be out of sync)

For this reason, I think that each update should have a serial number
attached to it, to be issued by the primary database (yes, this smells
like DNS..). The secondary should not process update 4 if it hasn't
done update 3.

Full database copies should maybe carry the same serial number as the last
update. For database version 100, all updates with serial <= 100 can be
deleted because they are already included.

I can think of 2 situations that need special attention:
1. The secondary becomes temporary disconnected (network outage).
   In this case, newer updates (send after the network is restored)
   will arrive before the older ones arrive (which are queued for
   the next sendmail queue run). The secondary will synchonize
   as soon as the older updates are in.
2. An update message is lost (forever).
   In this case, the secondary will never catch up. One can think of
   making some alarm bells that ring if the serial number difference
   between updates and the secondary gets over a certain threshold.
   In this scheme, this would be the only time that one would need to
   install a full newer copy of the database.
   Given the reliability of email, I think this will not happen 
   except under extreme circumstances. I think we should try this
   before people start investing effort in more elaborate schemes.

On the scaling issue, I think that having 10 a 20 routing registries
will certainly work with this scheme (compare with mailing lists
with 100s of subscribers). If things really get out of hand, then
one might think of using some reliable multicasting scheme, like
Van Jacobsen's WB has.

Just my 'stuiver' worth.. (we don't have cents in the Netherlands anymore ;-)

Geert Jan




-------- Logged at  Fri Mar 3 14:38:48 MET 1995  ---------