[ris-int] Re: [Fwd: backup bottlenecks and possible improvements]
Rene Wilhelm wilhelm at ripe.net
Tue Sep 6 17:24:13 CEST 2005
Hi Arife, >> I had a quick look at the raw data, and I think we can save a *lot* of >> space. A comparison tells me that if we convert from gzip to bzip2, we >> would save something like 25% of the space. This is only 100 Gbyte or >> so, but nice to have. > > I think it's a good idea. We can convert from gzip to bzip2. Few > applications that use rawdata need to be changed to use bzip2. > If you agree to use bzip2, I can proceed, and do the change. I have no principal objections against a switch to bzip2 but it has to be done in a coordinated way: RISwhois depends on raw data !!! Two changes are needed: 1. libbgdump MUST support bzip2 (dynamically, user shouldn't have to worry about it, just pass any filename to bgpdump_open and that routine should do the magic of reading the gzip or bzip file in the correct way). 2. riswhoisd MUST recognise .bz2 as a valid extension for RIS raw data files when searching archive dirs for dumps Currently, I don't know when I could work on #2, but as it depends on #1 anyhow, I'll first wait for a new libbgpdump to be ready before worrying about riswhois code itself. Cheers, -- Rene > gerard wrote: > > >We did not anticipate that the diskusage for the former NP-services > >would have such an enormous impact on the EMC. Approx. 1 GB is solely > >taken by the full backup for RIS, TTM and dnsmon. :-)) no surprise, NP projects were always data hungry :P
[ Ris-int Archives ]