[routing-wg] RPKI validator 3: initial sync regression?

Hello,
we also observe problems on our installations with never versions.

After very long period (hour+), all repositories loaded in 397.

But after few hours of running, database file quickly grew up (13 GB
total size, with more than 9 GB alocated within ~half hour). This
problem was introduced somewhere between versions 386 and 397.

Based on local observations, we returned to 377 for now. It seems,
there're multiple regressions in new versions...

- Daniel

On 3/20/19 12:54 PM, Job Snijders wrote:
> Hi all,
> 
> I can confirm that 377 was the last release that was... usable.
> 
> I tried 394 and yesterday's 397. Version 397 had only managed to load
> RIPE + APNIC after 30 minutes, with no indication when the other
> repositories would finish.
> 
> Kind regards,
> 
> Job
> 
> On Tue, Mar 19, 2019 at 04:12:07PM +0100, Nathalie Trenaman wrote:
>> Hi nusenu,
>>
>> We are looking into the situation with the validator as it seems to be
>> problematic since release 378. Here are our findings so far:
>>
>>
>> 	1	Version         Jar creation time               Loading from scratch
>> 	2	———————————————————————————————————
>> 	3
>> 	4	366             Jan 28, 11:47                   5 minute’ish
>> 	5	370             Jan 28, 18:40                   5 minute’ish
>> 	6	376             Jan 31, 14:15                   5 minute’ish
>> 	7	377             Feb 01, 13:30                   5 minute'ish
>> 	8	378             Mar 06, 14:23                   Stuck after 10 mins, only pending APNIC
>> 	9	381             Mar 06, 18:30                   Stuck after 10 mins, AFRINIC, LACNIC, APNIC done
>>
>> For now, I recommend to revert back to release 377 while we work on fixing the issue. Unfortunately, this build did not include the https fix for the RIS dumps.
>> There is an issue in GitHub: https://github.com/RIPE-NCC/rpki-validator-3/issues/77 <https://github.com/RIPE-NCC/rpki-validator-3/issues/77>
>>
>> Regarding the git tags and/or change logs, you are right, we will include them in next releases.
>>
>> We’ll keep the mailinglist and GitHub posted.
>>
>> Thanks,
>> Nathalie Trenaman
>> RIPE NCC
>>
>>> Op 18 mrt. 2019, om 18:17 heeft nusenu <nusenu-lists at riseup.net> het volgende geschreven:
>>>
>>> Hi,
>>>
>>> lately with validator 3 version > 370 it felt like
>>> rpki-validator 3 is less reliable and taking
>>> longer to download RPKI repository data (especially RIPE's)
>>> when starting for the first time when compared
>>> to older version (313) where it probably took less than
>>> 5 minutes after starting it to have the data for
>>> all regions. I reproduced this consistently.
>>>
>>> Can you think of anything that caused this?
>>> Does anyone else see this as well?
>>>
>>> Unfortunately I didn't find any changelogs
>>> and it is not obvious what version relates to
>>> what git commit since there are no git tags
>>> in the github repo.
>>>
>>> It would be great to have changelogs or
>>> git tags.
>>>
>>> thank you,
>>> nusenu
>>>
>>>
>>> --
>>> https://twitter.com/nusenu_
>>> https://mastodon.social/@nusenu
>>>
>>>
>>>
>>
> 
> 
>