[db-wg] "status:" attribute values in the RIPE database

Dear Colleagues,

We have now run the cleanup job to uppercase all "status:" attribute values, and updated 23,890 objects in total.

Regards
Ed Shryane
RIPE NCC


> On 24 Aug 2021, at 13:39, Edward Shryane via db-wg <db-wg at ripe.net> wrote:
> 
> Dear Colleagues,
> 
> After starting work on the implementation, we discovered that "status:" values are *already* automatically converted to uppercase on update. This was implemented in Whois release 1.73.1 and deployed on 27th May 2014, but existing non-uppercase values were not cleaned up at the time.
> 
> I propose simplifying the implementation plan as follows:
> 
> (1) Update Whois business rule validation (already implemented)
> 
> When a non-uppercase "status:" value is converted to uppercase, a global informational message is added to the response:
> 
> 	***Info:    Value %s converted to %s
> 
> (2) Cleanup existing non-uppercase "status:" values in the RIPE database
> 
> On next *Monday 30th August*, we will run a cleanup script to convert all existing non-uppercase "status:" values to uppercase.
> 
> We found approximately 21,492 out of 4,217,303 inetnum objects; 2,577 out of 655,856 inet6num objects; 0 out of 37,056 aut-num objects, with a non-uppercase "status:" value.
> 
> As only the case of the "status:" value is changed, the objects remain syntactically the same. We will *not* send notification emails to maintainers for the change, apart from notifying the db-wg. The "last-modified:" value will not change, but a new object version will be created (so the change will be visible in the version history).
> 
> All changes will be visible in NRTM as the objects are updated. The change can be considered a NOOP.
> 
> All changes will be visible in that night's database dump and split files.
> 
> We expect the cleanup script to take approximately an hour to complete all updates. 
> 
> Cleanup script updates will be applied cooperatively between normal updates, we do not expect any interruption to Whois updates.
> 
> Regards
> Ed Shryane
> RIPE NCC
> 
> 
> 
>> On 28 Jul 2021, at 17:15, Edward Shryane via db-wg <db-wg at ripe.net> wrote:
>> 
>> Dear Colleagues,
>> 
>> Thanks to the co-chairs for declaring a consensus, the DB team will start working on the implementation.
>> 
>> The implementation plan consists of two parts:
>> 
>> (1) Update Whois business rule validation
>> 
>> We will add a business rule to require the "status:" attribute value in inetnum, inet6num and aut-num objects to be completely in UPPERCASE.
>> 
>> This business rule will apply when creating or updating these object types.
>> 
>> If the "status:" value is not completely in uppercase, the software will automatically convert the value to uppercase and add a warning in the update response, immediately following the "status:" attribute: "***Warning:   The status value was converted to uppercase". The update will not fail if this happens.
>> 
>> We plan to deploy this change in the next Whois release (provisionally, version 1.102).
>> 
>> (2) Cleanup existing non-uppercase "status:" values in the RIPE database
>> 
>> Following the Whois release, we will run a cleanup script to convert all existing non-uppercase "status:" values to uppercase.
>> 
>> We found approximately 21,492 out of 4,217,303 inetnum objects; 2,577 out of 655,856 inet6num objects; 0 out of 37,056 aut-num objects, with a non-uppercase "status:" value.
>> 
>> As only the case of the "status:" value is changed, the objects remain syntactically the same. We will not send notification emails to maintainers for the change, apart from notifying the db-wg. The "last-modified:" value will not change, but a new object version will be created (so the change will be visible in the version history).
>> 
>> All changes will be visible in NRTM as the objects are updated. The change can be considered a NOOP.
>> 
>> All changes will be visible in that night's database dump and split files.
>> 
>> We expect the cleanup script to take approximately an hour to complete all updates. 
>> 
>> Cleanup script updates will be applied cooperatively between normal updates, we do not expect any interruption to Whois updates.
>> 
>> Please let me know if you have any questions.
>> 
>> Regards
>> Ed Shryane
>> RIPE NCC
>> 
>> 
>>> On 27 Jul 2021, at 15:59, denis walker <ripedenis at gmail.com> wrote:
>>> 
>>> Colleagues
>>> 
>>> The chairs recognise a consensus to force "status:" values to
>>> uppercase on updates and for the RIPE NCC to perform a one time update
>>> across the database to convert existing "status:" values to uppercase
>>> where necessary.
>>> 
>>> We ask the RIPE NCC to go ahead with this process and inform the
>>> community of their implementation plans.
>>> 
>>> regards
>>> William & denis
>>> co-chairs DB-WG
>>> 
>>> On Mon, 7 Jun 2021 at 11:09, Edward Shryane via db-wg <db-wg at ripe.net> wrote:
>>>> 
>>>> Dear colleagues,
>>>> 
>>>> In Remco van Mook's presentation to the Address Policy WG at RIPE 82: "What Colour Is My IP(v4) space? PI Addresses and LIRs", he pointed out that resource "status:" attribute values may not always be all UPPERCASE.
>>>> 
>>>> This is because the RIPE database is case-insensitive and case preserving in general, meaning a "status:" attribute value is the same regardless of the case.
>>>> 
>>>> However, this may make parsing this value more difficult as the case should be ignored.
>>>> 
>>>> Should the DB team implement a rule to always force the "status:" attribute value to uppercase, for consistency?
>>>> 
>>>> Regards
>>>> Ed Shryane
>>>> RIPE NCC
>>>> 
>>>> 
>> 
>> 
> 
>