[atlas] RIPE Atlas SMTP Measurement

No legitimate incoming SMTP traffic comes from IPs that abort connections
to my mx prior to attempting to deliver mail, so however I choose to
declutter my log files has absolutely zero real world impact in
deliverablity of legitimate incoming mail. Nor does it hurt anybody at the
other end whatever the connect-and-do-nothing software at the other side.

On Tue, Sep 20, 2022, 12:37 PM <ripe.net at toppas.net> wrote:

> >> because common configurations of fail2ban [...] absolutely will ban
> your IP [...] after multiple connection attempts and no successful mail
> transfer
>
> I would consider this a heavy misconfiguration. Please explain to me what
> incomplete SMTP connections have in common with spammers, virus/worm/trojan
> compromised hosts or open relay searching bots. Those bad senders WANT to
> *successfully* deliver mails to you. They will never abort the connection
> on purpose. For example: bots which search for open relays ALWAYS try to
> send mails with a foreign sender and recipient domain. That's how you
> discover them. But as suggested, the Atlas SMTP check should not send
> E-Mails at all, not even send MAIL FROM: or RCPT TO: command.
>
> You will not achieve mitigation of inbound spam/malware/phishing traffic
> by blocking IP addresses of hosts from incomplete SMTP sessions. Usually,
> incomplete SMTP sessions indicate a misconfiguration. For example: forced
> TLS enabled, but expired certificate or no matching cipher suites. But that
> is no reason to ban the senders! I think you have to go a little bit deeper
> in your logs and consider why mailtransfer was not successfull, before
> blocking ip addresses.
>
> I am no expert for fail2ban, but as far is i know, i searches for failed
> login attempts. So that affects mostly authenticated SMTP connections
> (client E-Mail submission on tcp/465 or tcp/587), right? This should not
> concern us here.
>
> I work with enterprise mailgateway solutions for years (mostly
> Proofpoint), but i have never encountered what you describe.
>
> Reject or throttle because of too much connections at the same time? Yes.
> Reject or throttle because of too much non-existing recipient adresses?
> Yes.
> Reject or throttle because both sender and recipient domain is foreign?
> Yes.
> Reject or throttle because of bad reputation (known spammer or TOR exit
> node ip address)? Yes.
>
> But not because of incomplete SMTP connections. From my point of view, I
> can not confirm that this common behaviour.
>
> BR,
> Simon
>
>
> On 20.09.22 19:22, Eric Kuhnke wrote:
>
> I would discourage anyone from relying upon the data from "probing" the MX
> and SMTP daemons for a domain name no matter what port they run on, because
> common configurations of fail2ban used with postfix and others absolutely
> will ban your IP at the host operating system level (iptables or other)
> after multiple connection attempts and no successful mail transfer or
> authentication.
>
> a probe of smtpd will look not much different from the many things out
> there on the internet already maliciously probing smtpd trying to find open
> relays.
>
>
>
>
> On Tue, 20 Sept 2022 at 09:22, Simon Brandt via ripe-atlas <
> ripe-atlas at ripe.net> wrote:
>
>> Hi Michel,
>>
>> Currently, HTTP and SSL are separate measurements. But for SMTP it will
>> probably not work this way... Encryption is not mandatory for SMTP and thus
>> negotiated between client and server every time on port 25. Ports 465 and
>> 587 are used for Client Email Submission. You could run some measurements
>> for these ports as well, but for the beginning, i would focus on
>> server2server communication.
>>
>> So here's what i think:
>>
>> What we could measure:
>> - General reachability/availability of the e-mail service
>> - Response time of the remote server: time between connection establish
>> and first SMTP response (220 service ready)
>> - Which enhanced status codes are offered by the server?
>> - Forward/Reverse DNS matching?
>> - SMTP banner matching the DNS name?
>>     - if not: what is it?
>> - Does the remote server offer encryption (250-STARTTLS)
>> - Which cipher settings are offered by the server (SSL/TLS Version, Key
>> Exchange Algorithms, Encryption Algorithms, Hashing Algorithms)
>>     - alternatively: Which cipher setting has been negotiated between
>> probe and server?
>> - Can we successfully establish a TLS connection?
>> - Certificate Check: is the server-certificate valid and does it match
>> the hostname?
>> - Forced Encryption: MTA-STS available and 'enforced' or 'testing' or
>> 'report'?
>> - Forced Authentication: DANE available and check successfull?
>>
>>
>> What we should not do:
>> - send MAIL FROM: command
>> - send RCPT TO: command
>> - send DATA command
>> - measure e-mail delivery/roundtrip time, etc.
>> - Sending e-mails at all
>>
>> The Atlas probe should quit the connection after the data is collected.
>> An actual e-mail should not be send. The target mailserver would count this
>> session as "incomplete" or "aborted" which is totally fine. If someone
>> would want to monitor what happens after a mailserver has successfully
>> accepted a testmail, he should better use a monitoring service/solution. We
>> measure the INTERnet, not what comes after (Intra) :)
>>
>> I don't expect any "spam" problems. Since the Atlas probes wouldn't send
>> e-mails, there's nothing a spamfilter could analyze. The only thing that
>> could theoretically happen, is that the probes ip addresses are flagged as
>> bad by services like Spamhaus etc. and thus be listed on DNSBL/IPBL, but i
>> really don't see a reason why that should happen. There wouldn't be any
>> activity originating from the probes which could be classified as bad. IP
>> addresses are normally only blacklisted, if they send unwanted mails like
>> spam. Also: there are a lot of "check you mailserver" or "check your
>> SSL/TLS" websites. The RIPE Atlas probes would behave the same way. No big
>> deal.
>>
>> Maybe we can think of an "extended" SMTP measurement where RIPE Atlas
>> sends actual e-mails, but that would require (in my opinion), that the
>> person who is creating the measurement somehow provides proof, to be the
>> owner of the target mailserver.
>>
>>
>> BR,
>> Simon
>>
>>
>> On 15.09.22 12:03, Michel Stam wrote:
>>
>> Hello Simon,
>>
>> Thank you for the suggestion.
>>
>> I have a couple of questions to get a better idea:
>>
>>    - Can you maybe describe what a SMTP measurement would look like?
>>       - Simple EHLO/HELO
>>       - Sending an email to a designated address (which would then
>>       validate that the SMTP server is capable of relaying etc.)
>>    - How would DNSBL or other spam prevention techniques fit into this?
>>    - What would the result be?
>>       - Delay until mail received
>>       - Response time by the actual mail server
>>       - Using the Received: headers to get a “traceroute” like result.
>>    - What about the more uncommon ports such as 565 (SMTP+SSL/TLS) or
>>    587 (mail submission port).
>>    - How can we prevent this implementation from having RIPE Atlas be
>>    identified as a spam bot network?
>>
>>
>> Best regards,
>>
>> Michel
>>
>> On 3 Sep 2022, at 14:48, Simon Brandt via ripe-atlas <ripe-atlas at ripe.net>
>> wrote:
>>
>> Hello,
>>
>> i'd like to start a discussion about having a RIPE Atlas SMTP
>> measurements.
>> First of all: yes, i know there's a big obstacle for such a measurement
>> type. A lot of probes are deployed using enduser internet-connections like
>> dsl, cable, etc. with dynamic/eyeball IP addresses. Those IP spaces are
>> usually blocked by SMTP servers as approach to reduce spam mails. For
>> Example: by using blocklists like Spamhaus PBL. So a proper SMTP
>> measurement wouldn't work.
>>
>> BUT we could have an easy way for RIPE Atlas probe hosters to signalize,
>> that their probe is eligible for SMTP measurements:
>>
>> Step 1: enable "Simple DNS Entry"
>> Step 2: create a matching reverse DNS record for the probes IP address
>>
>> Everybody who is able so configure a reverse DNS record for his probes IP
>> address, is most likely using a non-dynamic/non-home ip address space e.g.
>> a datacenter or office network. If an ISP provides the option for his
>> customers to configure a reverse DNS record, it's most likely a
>> "business-customer" subnet which can be used to run mailservers. After Step
>> 1+2 are done, the RIPE Atlas platform would easily be able to verify if
>> forward-confirmed reverse DNS is successful, and if so, automatically
>> enable that probe for SMTP measurements. Alternatively: probe hosters
>> choose their own Forward-confirmed reverse DNS name and submit it on the
>> RIPE Atlas website.
>>
>> Also: if we would have STMP measurements, forward-confirmed reverse DNS
>> should be mandatory for anchors, or is it already?
>>
>> Why should we have SMTP measurements?
>>
>> Besides general reachability checks, we could also evaluate SMTP response
>> codes. But the most important thing for me is this: the SMTP protocol is
>> old. Very old. From a security point of view, it's absolutely outdated.
>> Most security features have been added years after the initial RfC. Thus,
>> those security features are optional. Mandatory SMTP encryption is not
>> provided by the SMTP RfC. So both sides have to signalize, that they are
>> capable of encryption using the STARTTLS command. An attacker
>> (man-in-the-middle) can perform a downgrade attack by suppressing the
>> STARTTLS command. So both sides are forced to fallback and communicate
>> unencrypted. RIPE Atlas would be a really good tool to identify such
>> attacks, by monitor/measure the (enhanced) status codes of a target.
>>
>> But there's more!
>> I see a two-sided model here. Either use the RIPE Atlas SMTP measurements
>> to monitor/measure your own mailserver by alot of other RIPE probes out
>> there, OR probe hosters could run SMTP measurements originating from their
>> own probe to find out, if their own IP address is currently blocked by
>> other mailservers.
>>
>>
>> What do you think?
>>
>>
>> BR,
>> Simon
>> --
>> ripe-atlas mailing list
>> ripe-atlas at ripe.net
>> https://mailman.ripe.net/
>>
>>
>>
>> --
>> ripe-atlas mailing list
>> ripe-atlas at ripe.net
>> https://mailman.ripe.net/
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: </ripe/mail/archives/ripe-atlas/attachments/20220920/3bbfc682/attachment.html>