[db-wg] Options for extending "abuse-c:"

Dear Aleksi

We do seem to be drifting a long way off topic as far as abuse-c is 
concerned. But since you mentioned ideology I thought I would reply with 
some thoughts. I am not saying I have all the answers, or even any of 
the answers. I am not even saying all my ideas are practical or doable. 
I just want to raise some questions in people's minds about the 
complexity of the RIPE Database and the ongoing problems that so many 
new users have when trying to understand how to use it.


On 07/05/2014 15:22, Aleksi Suhonen wrote:
> Hello dbwg,
>
> On 05/07/2014 01:52 PM, Denis Walker wrote:
>> Sorry for another long email, but there really are a lot more discussion
>> points about abuse-c implementation than you think...so I highlighted
>> the important bit which was the last paragraph below and moved it to 
>> here:
>
> Thank you.
>
>> On 07/05/2014 03:27, Aleksi Suhonen wrote:
>>> I find this suggestion clumsy. It adds hard to parse extraneous
>>> information to simple objects. The organization object for a very
>>> large organization would become unmanageable and unintelligible 
>>> quickly.
>
>> Who do you believe is going to parse this object for this information?
>
> Anyone who *wants* to? Isn't that what open source, open data and open 
> interfaces are for?

The main goal of the database team is to manage, maintain and develop a 
modern, efficient, secure database service with features and interfaces 
that allows members to easily perform their daily tasks that interact 
with this database and provide access to the operational and 
registration details to the public. Although we make the software 
available as open source and accept patches from members who request 
additional features or fix known bugs, this is not the main goal.

>
>> The RIPE NCC already has an Abuse Finder tool which can be accessed
>> directly or via RIPEstat. As I said in the last paragraph of my article,
>> people should start to move away from the old fashioned idea of digging
>> directly into the RIPE Database themselves to find data, parse it and
>> interpret it. If you need information the RIPE NCC will provide web
>> tools and API calls to supply that information. We will  do all the
>> digging, parsing and interpretation for you.
>
> Oh dear, I'm feeling a huge ideological rift has developed here. Let 
> me get this straight:
>
> What you want is to hide away the whole RIPE database and provide a 
> point and click interface which only dispenses those morsels of 
> information you think people should have?

This is definitely not what I want, nor is it what I said. But it does 
no harm to occasionally ask questions of this 15 year old design and see 
if it still works "for everyone". In terms of hiding data away, it was 
agreed to hide the MD5 hash for security reasons. Let me ask you a 
question....does it benefit you, or me or the public that everyone can 
see your MNTNER object? What data exists in your MNTNER that I need to 
know? Do I need to know who, in your organisation, receives 
notifications of failed updates because of authorisation failures? Do I 
need to know how you have partitioned the management of your resources 
within your organisation?

The database contains two basic types of data - public 
operational/registration data and a whole lot of data concerning the way 
you manage that operational/registration data. This management data is 
all about you and your organisation and the way you arrange your data. 
Does all of that really need to be public? Does any of the management 
data need to be public?

>
> This may be fine for casual end users. However, I object at the idea 
> of having it as the central design paradigm of the whole RIPE db.

There may be many more database users than you think that would love to 
have simple interfaces to this data/information. We run a full 1 day 
training course to teach people only the basics of using this database. 
Why - because it is not intuitive and the design is over complicated.

>
>> In the end it should not matter to you where this data is stored in the
>> database. You deal in information, we deal in data storage and
>> retrieval. To be honest we don't even need to put this data into any
>> object. We can just store it as meta data associated with your
>> organisation. As long as we provide you with tools to view and manage
>> the information and for the public to find what they need....leave the
>> storage to us.
>
> Um ... and the ISPs who use some integrated provisioning and IPAM 
> software to automatically manage all their RIPE data should go and 
> hire more secretaries to point and click at the wizard instead?

Unless this provisioning software was written in the 90s, I am sure it 
can handle API calls.

>
> How do I get back to the 1990s where all the other people who think 
> like me used to live?
>
>>> I would much rather like to see a new inetnum and inet6num object
>>> status "INFORMATIONAL" that only requires authorization of the
>>> immediately larger enclosing inet(6)num object, and doesn't represent
>>> an assignment or allocation at all. Such objects can then be used to
>>> redirect technical, administrative and abuse contacts to the proper
>>> places, as well as present their own remarks and descriptions.
>
>> I believe this is going in completely the wrong direction. This means
>> creating more 'management' objects and replicating even more data all
>> over the database. We already have 3.8 million INETNUM objects all with
>> a mandatory admin-c and tech-c. That is 7.6M bits of replicated data! We
>> only have 10k members. These members manage the majority of the end user
>> resources as well as their own networks. What this database is really
>> missing is inheritance. Most of this management information could be
>> stored with your organisation, as the abuse-c is, and then inherited by
>> most of the operational data.
>
> This line of reasoning seems to lead to completely removing all 
> inetnums and simply listing all address space in the organization 
> object instead?

I am referring to separating out the operational data from the 
management of that data. The management data does not need to be public 
and in most organisations a simple set of management data can easily be 
inherited by all their resources.

Web forms 'AND' API calls can return this data. It could be returned in 
many ways....even re-composed into RPSL objects.

>
>> Just did some quick stats...we have 7.4M objects in the database and
>> 1.96M unique nic-hdls used in the admin-c, tech-c and zone-c attributes.
>> We know some large users have a business model to create a nic-hdl for
>> every customer. They certainly account for a few hundred thousand of
>> these nic-hdls. So probably we have about 1.5M nic-hdls replicated over
>> 7.4 million objects. That is a lot of data duplication.
>
> What if, instead, you just make the contact attributes of most object 
> types optional? If they are missing they get inherited from either the 
> organization or the maintainer object. And then to clean up the 
> database, you can remove all those contact attributes that are the 
> same as what inheritance would yield.

This is one way of starting to separate management data from operational 
data. But you would still need management tools to see an overview of 
where you have added the optional attributes. In a large network you 
could easily miss where you have added them and be working with the 
wrong contact. It might still be better to group the additional 
attributes together, which would make it easier to manage for those who 
prefer to look at RPSL objects themselves instead of using purpose made 
tools.

>
> As an unexpected added bonus, you will gain all the problems of 
> multiple inheritance too! High five for modern concepts!
>
> By the way, am I the only one that thinks 10 million RIPE objects is 
> not a lot of data in 2014?

It is not a question of how many objects there are in the database. 
There are no tools to view the entire set and there is no use case for 
it. What matters is what (not how many) objects your organisation has, 
what is the relationship between them, where have you put optional data 
or how is it inherited, how do you manage them and how do you view your 
data. Also for the routing registry you need to be able to view a 
collection of operational data across many users and to be able to 
manipulate that data.

We have been asked to look at server side expansion of set objects. This 
means we interpret your request, manipulate data and provide you with a 
view of what you want....not a stream of raw RPSL objects. Do you think 
it is a bad thing that we don't just give you a 90s stream of raw data 
objects and let you do all the parsing and manipulation yourself?

There was community consensus that a default query for address space 
will return the abuse contact details. This is another view of 
information that does not include raw RPSL objects. So the precedence 
already exists for providing views of information directly from the RIPE 
Database.

Regards
Denis

>
> Yours,
>