[dns-wg] Analysis of NSD

Hi Jørgen,

On Oct 28, 2004, at 14:09, Jørgen Hovland wrote:

>>> and different replies depending on the country origin of the source 
>>> ip of the querying nameserver/client.
>>
>> Oops.
>>
>> What's the justification for that? And what about maintaining DNS 
>> coherency? Any ideas on how to make this work with DNSSEC (I realize 
>> you're not doing DNSSEC today, but this would seem to be 
>> fundamentally incompatible with any DNSSEC use whatsoever).
>
> Load balancy.  A bit similar to what Akamai is doing. Regarding 
> dnsssec we aren't quite there yet and don't have a solution to it.

Hmm. Urgl. But I see your point (much as I don't like it).

I think such things are rather evil (i.e. I'd rather have intelligent 
clients making informed decisions that have just barely sentient 
servers making decisions on my behalf based on assumptions about my 
environment). But I can understand that there's a demand for that type 
of service given that the average client is not intelligent at all but 
rather downright retarded.

I want to see DNSSEC happen, so I'm naturally concerned when I see 
designs that I believe to be incompatible with DNSSEC, but I can 
understand that DNSSEC support has not been your most requested feature 
so far and sympathize with the lack of a solution.

>> What happens to updates (to the master) that occur *during* the 
>> reload? My guess is that they get added to "the tail" of the reload 
>> to ensure that no change is left out during the reload. But in that 
>> case it seems to me that you already have the zone sorted in 
>> "transaction order" and the only thing needed to steer around the 
>> complete reloads would be some sort of version stamp that is shared 
>> between slave and master. Doesn't really have to be the SOA serial, 
>> you can use whatever you want.
>
> The slave connects and registers with the master first, locks sql in 
> read-only mode, clears unprocessed zone change messages from the 
> master (there is about 0.001% chance of any zone change messages at 
> this stage anyway) and then reloads from sql. Changes sent by the 
> master during this stage will be held in a queue and not processed 
> before reload is finished. This should guarantee that the local zone 
> data is equal to the master. If the slave should die during the lock a 
> certain timeout would unlock it.
>
> We do a complete reload since it only takes 3 seconds. This is where 
> it becomes interesting.

Wait. Are you saying that a complete reload of the zone (where all the 
data moves from the master to the slave) takes 3 seconds? For how large 
a zone? Cannot be very large.

Or are you saying that the nameserver basically closes the connection 
to the DB backend and then reopens it to read the data fresh (i.e. 
there is massive data movement between nameserver and DB backend within 
the slave, but only DB syncronization magic goes over the wire from the 
master)?

Or are you saying that the slave does the sql reads over the wire from 
the DB (i.e. the SQL DB is not locally replicated on the slave)?

I know next to nothing about DB machinery when it comes to stuff like 
replication, so please excuse my ignorance.

>  I am quite confident that comparing SOA/zone changes would actually 
> take longer time. At least for us using SQL since a SQL query will 
> take prox 20ms before a reply is given.  Lets say 10% of the domains 
> were altered. This is a pretty high number though. You have to get the 
> new SOA's from SQL. Lets just say that this has already been done. 
> Now, 10% altered zones out of 30 millions equals to 1000 minutes in 
> latency only to deal with sql zone retrieve calls, not the processing 
> of the data. I am quite sure a raw dump would require less time and 
> less cpu resources by the sql server and perhaps even on the slave 
> depending

I agree to the efficiency of a raw dump. However, I'm really dense 
today so I don't really understand why you're using a number as high as 
30M *zones*. No offense, but no one ought to put that much 
infrastructure into any single system regardless of the underlying 
technology.
I think a more realistic example would be to look at 30K zones and 1 
minute. And furthermore I don't understand why it is not possible to 
parallellize those calls.

Especially since they not all go to the same master. Or perhaps they do 
in your case? Is it possible to have a slave slave multiple zones from 
different masters with multiple TCP sessions in different directions?

> on the size of each zone. If you only have a few large zones then of 
> course the result would not be the same. However if you have frequent 
> updates on these few large zones you would probably have to reload 
> everything anyway. You could always try reducing the amount of sql

I mostly agree. If you have a few large zones (typically TLDs) my guess 
would be that even with a rather high volume of changes most of the 
changes would concern a smaller part of the data and hence IXFRs still 
make sense as long as you're able to keep the transaction logs. I.e. I 
have no idea whatsoever about the change frequency of .co.uk for 
example, but I'd be really surprised if not more than 60% stayed 
unchanged for a year.

> calls by grouping them together, but that might look "ugly".  
> Nameservers doesn't usually lose connectivity anyway, but of course 
> they do time to time.

Exactly. And that's the situation that interests me. That "ordinary 
operation" works just fine doesn't surprise me at all.

> There is also a solution doing transaction logging when a slave gets 
> disconnected. We skipped this because it is easier to add and delete 
> new slaves without having to configure the master without. A 
> transaction log could also get very high if the slave was down for a 
> large amount of time and the implications generally about knowing if a 
> slave actually performed the change or not made us skip this.

This is exactly the reasoning behind how IXFR works. I.e. a slave can 
request an IXFR with all the transactions from version N until now, but 
the master alway has the right to respond with an AXFR. This way the 
master may "jettison" the transaction log if it grows too much to be 
convenient to keep.

As to knowing if a slave performed a change or not that is also taken 
care of by the SOA serial. So, since you don't have that (or something 
similar OOB wrt to DNS data as I suggested) I understand your 
reasoning.

Johan