[dns-wg] Analysis of NSD

----- Original Message ----- 
From: "Johan Ihrén" <johani at autonomica.se>
To: "Jørgen Hovland" <jorgen at hovland.cx>
Cc: <dns-wg at ripe.net>
Sent: Thursday, October 28, 2004 2:38 PM
Subject: Re: [dns-wg] Analysis of NSD


> Hi Jørgen,
>
> On Oct 28, 2004, at 14:09, Jørgen Hovland wrote:
>
>> We do a complete reload since it only takes 3 seconds. This is where it 
>> becomes interesting.
>
> Wait. Are you saying that a complete reload of the zone (where all the 
> data moves from the master to the slave) takes 3 seconds? For how large a 
> zone? Cannot be very large.

Yes it's a rather small db with only 10k domains. This number is of course 
growing so the implementation had to support our future demands.

>
> Or are you saying that the nameserver basically closes the connection to 
> the DB backend and then reopens it to read the data fresh (i.e. there is 
> massive data movement between nameserver and DB backend within the slave, 
> but only DB syncronization magic goes over the wire from the master)?
>
> Or are you saying that the slave does the sql reads over the wire from the 
> DB (i.e. the SQL DB is not locally replicated on the slave)?
>

All nameservers connect to the same SQL server (only the master has 
read-write). Master is always connected due to updates issued by web/asp 
connections etc. Slaves are only connected during startup and during a 
reconnect to the master. We could replicate the db but then we would have 
some more syncronisation issues and we really don't see the point doing this 
for now. We use replication only for backup purposes. If the sql server is 
down then the slaves wont reload during master reconnect right away. They 
will retry for a few times before giving up.


> I agree to the efficiency of a raw dump. However, I'm really dense today 
> so I don't really understand why you're using a number as high as 30M 
> *zones*. No offense, but no one ought to put that much infrastructure into 
> any single system regardless of the underlying technology.
> I think a more realistic example would be to look at 30K zones and 1 
> minute. And furthermore I don't understand why it is not possible to 
> parallellize those calls.
>

The 30M was just to show some large numbers. There won't be less domains in 
the future than now thats for sure. A 30K ns with the same 10% would only 
result in 1minute latency, but this is still a lot more than what a full 
dump would use including handling the data and everything to having a fresh 
copy.

> Especially since they not all go to the same master. Or perhaps they do in 
> your case? Is it possible to have a slave slave multiple zones from 
> different masters with multiple TCP sessions in different directions?

There can be only one to rule them all, but a slave can also be a master for 
other slaves again although we don't use this feature.


>> on the size of each zone. If you only have a few large zones then of 
>> course the result would not be the same. However if you have frequent 
>> updates on these few large zones you would probably have to reload 
>> everything anyway. You could always try reducing the amount of sql
>
> I mostly agree. If you have a few large zones (typically TLDs) my guess 
> would be that even with a rather high volume of changes most of the 
> changes would concern a smaller part of the data and hence IXFRs still 
> make sense as long as you're able to keep the transaction logs. I.e. I 
> have no idea whatsoever about the change frequency of .co.uk for example, 
> but I'd be really surprised if not more than 60% stayed unchanged for a 
> year.
>
>
> This is exactly the reasoning behind how IXFR works. I.e. a slave can 
> request an IXFR with all the transactions from version N until now, but 
> the master alway has the right to respond with an AXFR. This way the 
> master may "jettison" the transaction log if it grows too much to be 
> convenient to keep.

Sorry I should have said transaction log slash IXFR, not just transaction 
log. Anyway you are absolutely correct but we chose to skip this feature. 
IXFR is definately a good way to deal with zone synchronization (for 
existing zones as far as I am aware of). The dump implementation we use 
synchronizes everything including new and deleted zones.



Joergen Hovland ENK