[ris-int] Re: [Fwd: backup bottlenecks and possible improvements]
Henk Uijterwaal henk at ripe.net
Tue Sep 6 18:05:19 CEST 2005
At 17:24 06/09/2005, Rene Wilhelm wrote: >Hi Arife, > > >> I had a quick look at the raw data, and I think we can save a *lot* of > >> space. A comparison tells me that if we convert from gzip to bzip2, we > >> would save something like 25% of the space. This is only 100 Gbyte or > >> so, but nice to have. > > > > I think it's a good idea. We can convert from gzip to bzip2. Few > > applications that use rawdata need to be changed to use bzip2. > > > If you agree to use bzip2, I can proceed, and do the change. > >I have no principal objections against a switch to bzip2 but it has >to be done in a coordinated way: RISwhois depends on raw data !!! I agree, we should make the software aware of the change _and_ coordinate this carefully with our users, so that their tools don't break either. Henk >Two changes are needed: > >1. libbgdump MUST support bzip2 (dynamically, user shouldn't have > to worry about it, just pass any filename to bgpdump_open and > that routine should do the magic of reading the gzip or bzip > file in the correct way). > >2. riswhoisd MUST recognise .bz2 as a valid extension for > RIS raw data files when searching archive dirs for dumps > >Currently, I don't know when I could work on #2, but as it depends >on #1 anyhow, I'll first wait for a new libbgpdump to be ready >before worrying about riswhois code itself. > >Cheers, > >-- Rene > > > gerard wrote: > > > > >We did not anticipate that the diskusage for the former NP-services > > >would have such an enormous impact on the EMC. Approx. 1 GB is solely > > >taken by the full backup for RIS, TTM and dnsmon. > >:-)) > >no surprise, NP projects were always data hungry :P ------------------------------------------------------------------------------ Henk Uijterwaal Email: henk.uijterwaal(at)ripe.net RIPE Network Coordination Centre http://www.amsterdamned.org/~henk P.O.Box 10096 Singel 258 Phone: +31.20.5354414 1001 EB Amsterdam 1016 AB Amsterdam Fax: +31.20.5354445 The Netherlands The Netherlands Mobile: +31.6.55861746 ------------------------------------------------------------------------------ Look here junior, don't you be so happy. And for Heaven's sake, don't you be so sad. (Tom Verlaine)
[ Ris-int Archives ]