<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
(Coffee is to blame for the length of this)<br>
It's worth bearing in mind that the internet is a large place,
populated with large, larger, very large, enormous,
gargantuan....and extremely tiny, extremely budget-limited,
personal, private servers that cannot afford the charges for
non-productive traffic. The internet is not just Proofpoint or
Sendgrid or "Meta" or etc.; it is also vast numbers of people in
second- and third-world countries trying to engage with the world
and learn, and who literally can't afford excess, unproductive bytes
hitting their server that will cost them money.<br>
<br>
I used to be the sole systems admin for a site that sent variously
anywhere from a million to eleven million emails per day (yes, all
opt-in, legitimate email for our customers, just for the record), so
I've spent a fair bit of my life going zen analyzing logs for hours
and hours. I'm now retired and run my own personal mail and
webservers, on one t3.micro on AWS that costs me a few hundred bucks
over the course of three years commitment, and a free server on each
of GCE and Oracle, the former of which also costs me a few bucks a
month in traffic charges. <br>
<br>
I'm not poor, and can afford these charges. So I don't block any
RIPE Atlas traffic (to my knowledge), as I don't have (or need)
extreme restrictions on incoming traffic. I do occasionally sift
through my logs to find new pointless traffic that's 'annoying' my
systems, and may set something up to block them. I run a couple of
stratum one timeservers from home, part of the ntp pool project, and
regularly have to block nitwits (some of them ISP's themselves) who
hammer my little primary RPI with multiple connections per minute. I
have to block those malefactors simply because they interfere with
legitimate traffic. There are countless ways that 'internet
hooligans' and/or the clueless can cause harm with useless traffic.<br>
<br>
There are potentially millions of individuals trying to contribute
for the benefit of the internet who don't have the lack of concern
for costs that I have. Thus why there are so many Raspberry Pi
probes out there in non-first-world nations; and likely no small
number of tiny personal mailservers running on a shared 56k line,
the cost of which is _just_ bearable to maintain.<br>
<br>
There is no one-size-fits-all on the internet. It is common - and
very human - to view the internet and world through our own
perspective.<br>
<br>
There are other perspectives. <br>
<br>
Okay, that's my coffee-fueled bloviation for the day, and with that,
I acknowledge the significant digression involved...so I'll shut up
now.<br>
<br>
Cheers!<br>
<br>
<div class="moz-cite-prefix">On 9/20/2022 13:04 PM, Simon Brandt via
ripe-atlas wrote:<br>
</div>
<blockquote type="cite"
cite="mid:c4015363-cf3e-0993-192a-a3ed508fa644@toppas.net">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<div class="moz-cite-prefix">There's literally no danger for you
(recipient), if the sender terminates the connection before the
an e-mail has been successfully transmitted. No need to ban the
ip address. You would only risk to block a legitimate ip address
which might have trouble sending e-mails to you. If you want to
do this because you don't want to see such connections in you
logs that's fine. But i do not share your opinion, that this is
a common configuration.<br>
<br>
However, we have digressed from the topic.<br>
<br>
BR,<br>
Simon<br>
<br>
<br>
<br>
On 20.09.22 21:47, Eric Kuhnke wrote:<br>
</div>
<blockquote type="cite"
cite="mid:CAB69EHhTHq0uWpLX=RS0zOKmqv3zfCP3qUCYde09VXLMOCmokA@mail.gmail.com">
<meta http-equiv="content-type" content="text/html;
charset=UTF-8">
<div dir="auto">No legitimate incoming SMTP traffic comes from
IPs that abort connections to my mx prior to attempting to
deliver mail, so however I choose to declutter my log files
has absolutely zero real world impact in deliverablity of
legitimate incoming mail. Nor does it hurt anybody at the
other end whatever the connect-and-do-nothing software at the
other side. </div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Tue, Sep 20, 2022, 12:37
PM <<a href="mailto:ripe.net@toppas.net"
moz-do-not-send="true" class="moz-txt-link-freetext">ripe.net@toppas.net</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>
<div>>> because common configurations of fail2ban
[...] absolutely will ban your IP [...] after multiple
connection attempts and no successful mail transfer<br>
<br>
I would consider this a heavy misconfiguration. Please
explain to me what incomplete SMTP connections have in
common with spammers, virus/worm/trojan compromised
hosts or open relay searching bots. Those bad senders
WANT to <u>successfully</u> deliver mails to you. They
will never abort the connection on purpose. For example:
bots which search for open relays ALWAYS try to send
mails with a foreign sender and recipient domain. That's
how you discover them. But as suggested, the Atlas SMTP
check should not send E-Mails at all, not even send MAIL
FROM: or RCPT TO: command.<br>
<br>
You will not achieve mitigation of inbound
spam/malware/phishing traffic by blocking IP addresses
of hosts from incomplete SMTP sessions. Usually,
incomplete SMTP sessions indicate a misconfiguration.
For example: forced TLS enabled, but expired certificate
or no matching cipher suites. But that is no reason to
ban the senders! I think you have to go a little bit
deeper in your logs and consider why mailtransfer was
not successfull, before blocking ip addresses.<br>
<br>
I am no expert for fail2ban, but as far is i know, i
searches for failed login attempts. So that affects
mostly authenticated SMTP connections (client E-Mail
submission on tcp/465 or tcp/587), right? This should
not concern us here.<br>
<br>
I work with enterprise mailgateway solutions for years
(mostly Proofpoint), but i have never encountered what
you describe.<br>
<br>
Reject or throttle because of too much connections at
the same time? Yes.<br>
Reject or throttle because of too much non-existing
recipient adresses? Yes.<br>
Reject or throttle because both sender and recipient
domain is foreign? Yes.<br>
Reject or throttle because of bad reputation (known
spammer or TOR exit node ip address)? Yes.<br>
<br>
But not because of incomplete SMTP connections. From my
point of view, I can not confirm that this common
behaviour.<br>
<br>
BR,<br>
Simon<br>
<br>
<br>
On 20.09.22 19:22, Eric Kuhnke wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div>I would discourage anyone from relying upon the
data from "probing" the MX and SMTP daemons for a
domain name no matter what port they run on, because
common configurations of fail2ban used with postfix
and others absolutely will ban your IP at the host
operating system level (iptables or other) after
multiple connection attempts and no successful mail
transfer or authentication. <br>
</div>
<div><br>
</div>
<div>a probe of smtpd will look not much different
from the many things out there on the internet
already maliciously probing smtpd trying to find
open relays.</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Tue, 20 Sept 2022
at 09:22, Simon Brandt via ripe-atlas <<a
href="mailto:ripe-atlas@ripe.net" target="_blank"
rel="noreferrer" moz-do-not-send="true"
class="moz-txt-link-freetext">ripe-atlas@ripe.net</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px
0px 0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div>
<div>Hi Michel,<br>
<br>
Currently, HTTP and SSL are separate
measurements. But for SMTP it will probably not
work this way... Encryption is not mandatory for
SMTP and thus negotiated between client and
server every time on port 25. Ports 465 and 587
are used for Client Email Submission. You could
run some measurements for these ports as well,
but for the beginning, i would focus on
server2server communication.<br>
<br>
So here's what i think:<br>
<br>
What we could measure:<br>
- General reachability/availability of the
e-mail service<br>
- Response time of the remote server: time
between connection establish and first SMTP
response (220 service ready)<br>
- Which enhanced status codes are offered by the
server?<br>
- Forward/Reverse DNS matching?<br>
- SMTP banner matching the DNS name?<br>
- if not: what is it?<br>
- Does the remote server offer encryption
(250-STARTTLS)<br>
- Which cipher settings are offered by the
server (SSL/TLS Version, Key Exchange
Algorithms, Encryption Algorithms, Hashing
Algorithms)<br>
- alternatively: Which cipher setting has
been negotiated between probe and server?<br>
- Can we successfully establish a TLS
connection?<br>
- Certificate Check: is the server-certificate
valid and does it match the hostname?<br>
- Forced Encryption: MTA-STS available and
'enforced' or 'testing' or 'report'?<br>
- Forced Authentication: DANE available and
check successfull?<br>
<br>
<br>
What we should not do:<br>
- send MAIL FROM: command<br>
- send RCPT TO: command<br>
- send DATA command<br>
- measure e-mail delivery/roundtrip time, etc.<br>
- Sending e-mails at all<br>
<br>
The Atlas probe should quit the connection after
the data is collected. An actual e-mail should
not be send. The target mailserver would count
this session as "incomplete" or "aborted" which
is totally fine. If someone would want to
monitor what happens after a mailserver has
successfully accepted a testmail, he should
better use a monitoring service/solution. We
measure the INTERnet, not what comes after
(Intra) :)<br>
<br>
I don't expect any "spam" problems. Since the
Atlas probes wouldn't send e-mails, there's
nothing a spamfilter could analyze. The only
thing that could theoretically happen, is that
the probes ip addresses are flagged as bad by
services like Spamhaus etc. and thus be listed
on DNSBL/IPBL, but i really don't see a reason
why that should happen. There wouldn't be any
activity originating from the probes which could
be classified as bad. IP addresses are normally
only blacklisted, if they send unwanted mails
like spam. Also: there are a lot of "check you
mailserver" or "check your SSL/TLS" websites.
The RIPE Atlas probes would behave the same way.
No big deal.<br>
<br>
Maybe we can think of an "extended" SMTP
measurement where RIPE Atlas sends actual
e-mails, but that would require (in my opinion),
that the person who is creating the measurement
somehow provides proof, to be the owner of the
target mailserver.<br>
<br>
<br>
BR,<br>
Simon<br>
<br>
<br>
On 15.09.22 12:03, Michel Stam wrote:<br>
</div>
<blockquote type="cite"> Hello Simon,
<div><br>
</div>
<div>Thank you for the suggestion.</div>
<div><br>
</div>
<div>I have a couple of questions to get a
better idea:</div>
<div>
<ul>
<li>Can you maybe describe what a SMTP
measurement would look like?</li>
<ul>
<li>Simple EHLO/HELO</li>
<li>Sending an email to a designated
address (which would then validate that
the SMTP server is capable of relaying
etc.)</li>
</ul>
<li>How would DNSBL or other spam prevention
techniques fit into this? </li>
<li>What would the result be? </li>
<ul>
<li>Delay until mail received</li>
<li>Response time by the actual mail
server</li>
<li>Using the Received: headers to get a
“traceroute” like result.</li>
</ul>
<li>What about the more uncommon ports such
as 565 (SMTP+SSL/TLS) or 587 (mail
submission port).</li>
<li>How can we prevent this implementation
from having RIPE Atlas be identified as a
spam bot network?</li>
</ul>
</div>
<div>
<div> </div>
<div>Best regards,</div>
<div><br>
</div>
<div>Michel</div>
<div><br>
<blockquote type="cite">
<div>On 3 Sep 2022, at 14:48, Simon Brandt
via ripe-atlas <<a
href="mailto:ripe-atlas@ripe.net"
target="_blank" rel="noreferrer"
moz-do-not-send="true"
class="moz-txt-link-freetext">ripe-atlas@ripe.net</a>>
wrote:</div>
<br>
<div>
<div> Hello,<br>
<br>
i'd like to start a discussion about
having a RIPE Atlas SMTP measurements.<br>
First of all: yes, i know there's a
big obstacle for such a measurement
type. A lot of probes are deployed
using enduser internet-connections
like dsl, cable, etc. with
dynamic/eyeball IP addresses. Those IP
spaces are usually blocked by SMTP
servers as approach to reduce spam
mails. For Example: by using
blocklists like Spamhaus PBL. So a
proper SMTP measurement wouldn't work.<br>
<br>
BUT we could have an easy way for RIPE
Atlas probe hosters to signalize, that
their probe is eligible for SMTP
measurements:<br>
<br>
Step 1: enable "Simple DNS Entry"<br>
Step 2: create a matching reverse DNS
record for the probes IP address<br>
<br>
Everybody who is able so configure a
reverse DNS record for his probes IP
address, is most likely using a
non-dynamic/non-home ip address space
e.g. a datacenter or office network.
If an ISP provides the option for his
customers to configure a reverse DNS
record, it's most likely a
"business-customer" subnet which can
be used to run mailservers. After Step
1+2 are done, the RIPE Atlas platform
would easily be able to verify if
forward-confirmed reverse DNS is
successful, and if so, automatically
enable that probe for SMTP
measurements. Alternatively: probe
hosters choose their own
Forward-confirmed reverse DNS name and
submit it on the RIPE Atlas website.<br>
<br>
Also: if we would have STMP
measurements, forward-confirmed
reverse DNS should be mandatory for
anchors, or is it already?<br>
<br>
Why should we have SMTP measurements?<br>
<br>
Besides general reachability checks,
we could also evaluate SMTP response
codes. But the most important thing
for me is this: the SMTP protocol is
old. Very old. From a security point
of view, it's absolutely outdated.
Most security features have been added
years after the initial RfC. Thus,
those security features are optional.
Mandatory SMTP encryption is not
provided by the SMTP RfC. So both
sides have to signalize, that they are
capable of encryption using the
STARTTLS command. An attacker
(man-in-the-middle) can perform a
downgrade attack by suppressing the
STARTTLS command. So both sides are
forced to fallback and communicate
unencrypted. RIPE Atlas would be a
really good tool to identify such
attacks, by monitor/measure the
(enhanced) status codes of a target.<br>
<br>
But there's more!<br>
I see a two-sided model here. Either
use the RIPE Atlas SMTP measurements
to monitor/measure your own mailserver
by alot of other RIPE probes out
there, OR probe hosters could run SMTP
measurements originating from their
own probe to find out, if their own IP
address is currently blocked by other
mailservers.<br>
<br>
<br>
What do you think?<br>
<br>
<br>
BR,<br>
Simon<br>
</div>
-- <br>
ripe-atlas mailing list<br>
<a href="mailto:ripe-atlas@ripe.net"
target="_blank" rel="noreferrer"
moz-do-not-send="true"
class="moz-txt-link-freetext">ripe-atlas@ripe.net</a><br>
<a
href="https://mailman.ripe.net/"
target="_blank" rel="noreferrer"
moz-do-not-send="true"
class="moz-txt-link-freetext">https://mailman.ripe.net/</a><br>
</div>
</blockquote>
</div>
<br>
</div>
</blockquote>
<br>
</div>
-- <br>
ripe-atlas mailing list<br>
<a href="mailto:ripe-atlas@ripe.net" target="_blank"
rel="noreferrer" moz-do-not-send="true"
class="moz-txt-link-freetext">ripe-atlas@ripe.net</a><br>
<a
href="https://mailman.ripe.net/"
rel="noreferrer noreferrer" target="_blank"
moz-do-not-send="true"
class="moz-txt-link-freetext">https://mailman.ripe.net/</a><br>
</blockquote>
</div>
</blockquote>
<br>
</div>
</blockquote>
</div>
</blockquote>
<br>
<br>
<fieldset class="moz-mime-attachment-header"></fieldset>
</blockquote>
<br>
<div class="moz-signature">-- <br>
Paul Theodoropoulos<br>
<a href="https://www.anastrophe.com" moz-do-not-send="true">anastrophe.com</a></div>
</body>
</html>