[anti-abuse-wg] [Misc] Research project on blacklists

Hi,

I note that I am involved in one or more of the blacklists in your image.png
As you are aware, I have decided to take offense at the stated goal of
your research, (to "improve") as I know enough, to know, that you
do not understand all the protocols to generally make any such
oxymoronic statement as it regards certain aspects of blacklist
management and/or protocols. More so as it pertains specifically to
what I do. Even if your research would result in any measurable
improvement in any other (and operationally unknown to myself)
blacklist data it would still be far less meaningful as actual useful
abuse tech research, more specifically, I mean less of a shotgun and
more of a rifle. You also clearly do not understand how the differences
in protocols of your blacklist classification manages data and how this 
affects accuracy, as is demonstrated by the blocklists as they are reflected in
your own image.png, yet you want to improve data accuracy and you want
to be taken seriously. Anyway, this will need a review if it is to be
useful, accurate or not whitewashed paid for research. And no, I am not
keen on putting it on a spoon for you or adding any meat.

Andre

On Thu, 18 Jul 2019 15:33:39 +0200
Anushah Hossain <anushah at icsi.berkeley.edu> wrote:

> Apologies for my slow response - I have been traveling and also
> consulting with my team members on how best to respond (as you might
> have gleaned from my profile linked upthread, my own background is
> not in networking or security :)). I hope to share more thorough
> responses with you once the sun rises in their timezones.
> 
> >surprisingly, I haven't seen the request on any other lists that are
> >(a)  
> relevant and (b) open -- perhaps they and their project team are not
> especially well connected in this space :(
> 
> This is true. We were advised to share to RIPE and regional NOG
> mailing lists. Are there others you would have recommended?
> 
> > as John Levine already noted, the questionnaire seems somewhat
> > confused  
> as to whether it cares about routing issues (bogon lists, the Spamhaus
> DROP list etc) or spam filtering (bad domains, phishing feeds, botnet
> IPs etc etc)
> 
> Hm, I think we are interested in quite the range of blacklists. Here
> is a table of what my colleagues are monitoring:
> 
> [image: image.png]
> 
> >it also asked if internally generated lists were used, but seemed  
> curiously uninterested in anything other than if the answer to that
> was yes or no -- a missed opportunity I thought.
> 
> What would you have recommended probing here?
> 
> I do genuinely appreciate your discussion and patience. It is very
> interesting and useful for me to see what topics matter to you most
> and where we might have misdirected our attention. Just as
> background, we did pilot the survey with a smaller set of network
> operators and felt it had been straightforward to respond to, given
> their reactions. But as many of you have noted, the survey is rather
> general. I have been conducting interviews with those working in
> abuse prevention (even at some of the companies that have been
> mentioned upthread) to collect more specific anecdotes about how
> dynamic addressing has lowered the accuracy of certain feeds, for
> example, or how errors in geo-IP feeds affected them. The interviews
> allow for a bit more elucidation, but it has been difficult to
> recruit participants. Hence the survey.
> 
> All the best,
> Anushah
> 
> 
> 
> On Thu, Jul 18, 2019 at 2:36 PM Richard Clayton
> <richard at highwayman.com> wrote:
> 
> > In message , ac <ac at main.me> writes
> >  
> > >Mostly, what makes me very angry is the audacity  
> >
> > this does seem a reasonable list to ask for assistance on ... but
> > being around to answer questions promptly would be appropriately
> > polite
> >
> > surprisingly, I haven't seen the request on any other lists that
> > are (a) relevant and (b) open -- perhaps they and their project
> > team are not especially well connected in this space :(  though
> > there is a recent "anonymous" survey request about router
> > configurations on the NANOG list 
> > >and then the
> > >"anonymous"  
> >
> > the Qualtrics platform is available over Tor (unlike some online
> > survey platforms) so if you declined to answer the questions about
> > which AS and company you were associated with then there is a
> > substantial amount of anonymity available to you should you wish to
> > use it... 
> > >and I can already see the "findings" of this research...
> > >based on random anonymous, hidden and secret inputs....  
> >
> > that is a concern -- this type of questionnaire pretty much never
> > leads to high quality research directly (since there are
> > significant biases in who might choose to give replies and there is
> > scope for multiple responses from a single person, bots filling it
> > in etc)
> >
> > nevertheless as a starting point for qualitative research (rather
> > than quantitative) it can be very useful in allowing a researcher
> > to identify general trends in the answers and -- importantly -- to
> > help the researcher frame good research questions that are capable
> > of being investigated in more detail
> >
> > as John Levine already noted, the questionnaire seems somewhat
> > confused as to whether it cares about routing issues (bogon lists,
> > the Spamhaus DROP list etc) or spam filtering (bad domains,
> > phishing feeds, botnet IPs etc etc)
> >
> > it also asked if internally generated lists were used, but seemed
> > curiously uninterested in anything other than if the answer to that
> > was yes or no -- a missed opportunity I thought.
> >
> > --
> > richard                                                   Richard
> > Clayton
> >
> > Those who would give up essential Liberty, to purchase a little
> > temporary Safety, deserve neither Liberty nor Safety. Benjamin
> > Franklin 11 Nov 1755 
> 
>