[anti-abuse-wg] Question about spam to abuse inbox

On 21-02-2021 03:44 +0100, Cynthia Revström writes:
> Ronald,
> 
> Can you please stop attacking ideas (such as web forms) implying that
> they only have malicious use cases.
> 
> > I hold them responsible because they obviously
> > fail to have in place contractual clauses that would persuasively
> > deter this behavior on the part of their customers.
> 
> In many cases it is practically impossible to know if your customers
> are sending legit emails or spam without having people reporting it.
> As TLS is used in many cases now, the provider can't look at the
> network data to see what the customer is sending even on a technical
> level, disregarding any trust/potential legal issues.


> > The provider in question is a perfectly lousy coder and is thus
> > unable and/or unwilling to write code to parse emailed abuse
> > reports.
> 
> Hi, I am actually primarily a software dev and not a network
> engineer, it is not even close to as easy as you make it out to be.
> Sure you can have a regex to extract IP addresses and other messy
> things like that, but you can't be sure what that address is, it
> might be your customer, it might be the address they say you
> attacked, etc.
> My point here is that parsing free form text in this way without
> having a clearly defined structure is far from trivial.
> Also please stop assuming bad faith by saying that providers are
> "unwilling" to do this.
> If they could drastically lower the amount of manual work needed here
> with a bit of code, they absolutely would in almost all cases.

Hello Cynthia

I would say it's not as hard. Having the right tools helps a ton, but
not all companies understand that.
First of all, you want to automatically parse those reports using
ARF/X-ARF, as those are already machine parseable.
Then, you will have a lot of other reports is a mix of formats, that
you could be parsing separatedky. Although I would say that a naive
approach of “parse all IP addresses, if there is a single one in our
range, associate the report to that IP address” works in most cases.
Scanning for a few keywords (spam, DDoS, telnet, ssh…) should also
allow for an initial classification.

This is all very rough, and (as mentioned in the thread), you should
still have a human *look* at it, but could easily cut the work needed
in more than half.

If you receive those 200 Incident Reports, but they are already
classified as 185 of them relating to 203.0.113.7, you will probably
not need to evaluate all of them to conclude that there is something
bad going on with that customer.

Also, another point would be the number of clicks needed to take action
(e.g. in some systems you might need just 2-3 clicks to suspend the
customer resource and send them a warning, wheras in others you may
need a slow manual process).




> > And anyway, don't actual human beings need to look at these things,
> > in the end, in order to be able to react to each of them properly
> > and in a professional fashion? 
> 
> Web forms can have pros and cons, I am just going to take the case of
> a VPS/Dedicated server hosting company.
> 
> If the hosting company provides a web form, they can have a field
> where they explicitly ask for the offending IP address.
> This report could then automatically also be sent to the customer in
> question, because we shouldn't assume the customer is malicious, they
> might just have a bad config that made them a relay for example.
> This could make it so the report is acted upon sooner potentially as
> the hosting company might take a few days to reply but maybe the
> customer can act sooner.

It depends. In some cases, the customer is another victim. In others,
such as the customer having bought "paypa1.com", well, I think you
_should_ assume he is malicious. :)

It's not hard to figure out by a human. Yet you still need someone to
ascertain them.



> > A provider that is routinely receiving so many abuse reports that
> > it can barely keep up with them all has bigger problems that just
> > the manner in which abuse reports are received.
> 
> Due to the automated procedure by some providers for abuse reports,
> if I have one bad host sending spam, I might get an abuse report for
> every single email they receive, so even if it is just one customer I
> might wake up to 200 emails.
> But if I had a way to group it by sender IP address, that would be a
> lot more manageable.
> (this was just a hypothetical example) 
> 
> Now I absolutely agree that having an abuse email address that is
> acted upon in a reasonable amount of time (maybe a week or so) is
> still essential as the web forms aren't standardised or might rely on
> technology like captchas.
> But if you send me 200 emails about the same host in one day, I am
> probably still going to be mildly annoyed and I could see how this is
> actually unmanageable for larger providers.


Larger providers should have more people dedicated to handle abuse
reports. Unfortunately, it's a task working too many times with less
resources than would be required, if not even sabotaged by the higher
ups. I see how, on the surface, they may not understand the value
provided by a department "tasked with banning customers", but unless
taking action against the bad apples, they will find themselves their
ranges banned everywhere, and legitimate customers fleeing away.



> I think the true solution here is just to have a standard email
> template or similar so providers could easily and reliably parse it
> automatically (at least partially).
> just a very quick example that I didn't consider for more than a
> minute: the standard could be as easy as just beginning every report
> email with "abuse-host=192.0.2.20,192.0.2.21\n\n" and whatever other
> fields are needed.
> 
> -Cynthia

There are already some existing formats, and some companies already do
something similar, but if additional templates are found to be
required, I will be happy to see them developed.


Best regards


-- 
INCIBE-CERT - Spanish National CSIRT
https://www.incibe-cert.es/

PGP keys: https://www.incibe-cert.es/en/what-is-incibe-cert/pgp-public-keys

====================================================================

INCIBE-CERT is the Spanish National CSIRT designated for citizens,
private law entities, other entities not included in the subjective
scope of application of the "Ley 40/2015, de 1 de octubre, de Régimen
Jurídico del Sector Público", as well as digital service providers,
operators of essential services and critical operators under the terms
of the "Real Decreto-ley 12/2018, de 7 de septiembre, de seguridad de
las redes y sistemas de información" that transposes the Directive (EU)
2016/1148 of the European Parliament and of the Council of 6 July 2016
concerning measures for a high common level of security of network and
information systems across the Union.

====================================================================

In compliance with the General Data Protection Regulation of the EU
(Regulation EU 2016/679, of 27 April 2016) we inform you that your
personal and corporate data (as well as those included in attached
documents); and e-mail address, may be included in our records 
for the purpose derived from legal, contractual or pre-contractual
obligations or in order to respond to your queries. You may exercise
your rights of access, correction, cancellation, portability,
limitationof processing and opposition under the terms established by
current legislation and free of charge by sending an e-mail to
dpd at incibe.es. The Data Controller is S.M.E. Instituto Nacional de
Ciberseguridad de España, M.P., S.A. More information is available
on our website: https://www.incibe.es/proteccion-datos-personales
and https://www.incibe.es/registro-actividad.

====================================================================