<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
  </head>
  <body>
    (Coffee is to blame for the length of this)<br>
    It's worth bearing in mind that the internet is a large place,
    populated with large, larger, very large, enormous,
    gargantuan....and extremely tiny, extremely budget-limited,
    personal, private servers that cannot afford the charges for
    non-productive traffic. The internet is not just Proofpoint or
    Sendgrid or "Meta" or etc.; it is also vast numbers of people in
    second- and third-world countries trying to engage with the world
    and learn, and who literally can't afford excess, unproductive bytes
    hitting their server that will cost them money.<br>
    <br>
    I used to be the sole systems admin for a site that sent variously
    anywhere from a million to eleven million emails per day (yes, all
    opt-in, legitimate email for our customers, just for the record), so
    I've spent a fair bit of my life going zen analyzing logs for hours
    and hours. I'm now retired and run my own personal mail and
    webservers, on one t3.micro on AWS that costs me a few hundred bucks
    over the course of three years commitment, and a free server on each
    of GCE and Oracle, the former of which also costs me a few bucks a
    month in traffic charges. <br>
    <br>
    I'm not poor, and can afford these charges. So I don't block any
    RIPE Atlas traffic (to my knowledge), as I don't have (or need)
    extreme restrictions on incoming traffic. I do occasionally sift
    through my logs to find new pointless traffic that's 'annoying' my
    systems, and may set something up to block them. I run a couple of
    stratum one timeservers from home, part of the ntp pool project, and
    regularly have to block nitwits (some of them ISP's themselves) who
    hammer my little primary RPI with multiple connections per minute. I
    have to block those malefactors simply because they interfere with
    legitimate traffic. There are countless ways that 'internet
    hooligans' and/or the clueless can cause harm with useless traffic.<br>
    <br>
    There are potentially millions of individuals trying to contribute
    for the benefit of the internet who don't have the lack of concern
    for costs that I have. Thus why there are so many Raspberry Pi
    probes out there in non-first-world nations; and likely no small
    number of tiny personal mailservers running on a shared 56k line,
    the cost of which is _just_ bearable to maintain.<br>
    <br>
    There is no one-size-fits-all on the internet. It is common - and
    very human - to view the internet and world through our own
    perspective.<br>
    <br>
    There are other perspectives. <br>
    <br>
    Okay, that's my coffee-fueled bloviation for the day, and with that,
    I acknowledge the significant digression involved...so I'll shut up
    now.<br>
    <br>
    Cheers!<br>
    <br>
    <div class="moz-cite-prefix">On 9/20/2022 13:04 PM, Simon Brandt via
      ripe-atlas wrote:<br>
    </div>
    <blockquote type="cite"
      cite="mid:c4015363-cf3e-0993-192a-a3ed508fa644@toppas.net">
      <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
      <div class="moz-cite-prefix">There's literally no danger for you
        (recipient), if the sender terminates the connection before the
        an e-mail has been successfully transmitted. No need to ban the
        ip address. You would only risk to block a legitimate ip address
        which might have trouble sending e-mails to you. If you want to
        do this because you don't want to see such connections in you
        logs that's fine. But i do not share your opinion, that this is
        a common configuration.<br>
        <br>
        However, we have digressed from the topic.<br>
        <br>
        BR,<br>
        Simon<br>
        <br>
        <br>
        <br>
        On 20.09.22 21:47, Eric Kuhnke wrote:<br>
      </div>
      <blockquote type="cite"
cite="mid:CAB69EHhTHq0uWpLX=RS0zOKmqv3zfCP3qUCYde09VXLMOCmokA@mail.gmail.com">
        <meta http-equiv="content-type" content="text/html;
          charset=UTF-8">
        <div dir="auto">No legitimate incoming SMTP traffic comes from
          IPs that abort connections to my mx prior to attempting to
          deliver mail, so however I choose to declutter my log files
          has absolutely zero real world impact in deliverablity of
          legitimate incoming mail. Nor does it hurt anybody at the
          other end whatever the connect-and-do-nothing software at the
          other side. </div>
        <br>
        <div class="gmail_quote">
          <div dir="ltr" class="gmail_attr">On Tue, Sep 20, 2022, 12:37
            PM <<a href="mailto:ripe.net@toppas.net"
              moz-do-not-send="true" class="moz-txt-link-freetext">ripe.net@toppas.net</a>>
            wrote:<br>
          </div>
          <blockquote class="gmail_quote" style="margin:0 0 0
            .8ex;border-left:1px #ccc solid;padding-left:1ex">
            <div>
              <div>>> because common configurations of fail2ban
                [...] absolutely will ban your IP [...] after multiple
                connection attempts and no successful mail transfer<br>
                <br>
                I would consider this a heavy misconfiguration. Please
                explain to me what incomplete SMTP connections have in
                common with spammers, virus/worm/trojan compromised
                hosts or open relay searching bots. Those bad senders
                WANT to <u>successfully</u> deliver mails to you. They
                will never abort the connection on purpose. For example:
                bots which search for open relays ALWAYS try to send
                mails with a foreign sender and recipient domain. That's
                how you discover them. But as suggested, the Atlas SMTP
                check should not send E-Mails at all, not even send MAIL
                FROM: or RCPT TO: command.<br>
                <br>
                You will not achieve mitigation of inbound
                spam/malware/phishing traffic by blocking IP addresses
                of hosts from incomplete SMTP sessions. Usually,
                incomplete SMTP sessions indicate a misconfiguration.
                For example: forced TLS enabled, but expired certificate
                or no matching cipher suites. But that is no reason to
                ban the senders! I think you have to go a little bit
                deeper in your logs and consider why mailtransfer was
                not successfull, before blocking ip addresses.<br>
                <br>
                I am no expert for fail2ban, but as far is i know, i
                searches for failed login attempts. So that affects
                mostly authenticated SMTP connections (client E-Mail
                submission on tcp/465 or tcp/587), right? This should
                not concern us here.<br>
                <br>
                I work with enterprise mailgateway solutions for years
                (mostly Proofpoint), but i have never encountered what
                you describe.<br>
                <br>
                Reject or throttle because of too much connections at
                the same time? Yes.<br>
                Reject or throttle because of too much non-existing
                recipient adresses? Yes.<br>
                Reject or throttle because both sender and recipient
                domain is foreign? Yes.<br>
                Reject or throttle because of bad reputation (known
                spammer or TOR exit node ip address)? Yes.<br>
                <br>
                But not because of incomplete SMTP connections. From my
                point of view, I can not confirm that this common
                behaviour.<br>
                <br>
                BR,<br>
                Simon<br>
                <br>
                <br>
                On 20.09.22 19:22, Eric Kuhnke wrote:<br>
              </div>
              <blockquote type="cite">
                <div dir="ltr">
                  <div>I would discourage anyone from relying upon the
                    data from "probing" the MX and SMTP daemons for a
                    domain name no matter what port they run on, because
                    common configurations of fail2ban used with postfix
                    and others absolutely will ban your IP at the host
                    operating system level (iptables or other) after
                    multiple connection attempts and no successful mail
                    transfer or authentication. <br>
                  </div>
                  <div><br>
                  </div>
                  <div>a probe of smtpd will look not much different
                    from the many things out there on the internet
                    already maliciously probing smtpd trying to find
                    open relays.</div>
                  <div><br>
                  </div>
                  <div><br>
                  </div>
                  <div><br>
                  </div>
                </div>
                <br>
                <div class="gmail_quote">
                  <div dir="ltr" class="gmail_attr">On Tue, 20 Sept 2022
                    at 09:22, Simon Brandt via ripe-atlas <<a
                      href="mailto:ripe-atlas@ripe.net" target="_blank"
                      rel="noreferrer" moz-do-not-send="true"
                      class="moz-txt-link-freetext">ripe-atlas@ripe.net</a>>
                    wrote:<br>
                  </div>
                  <blockquote class="gmail_quote" style="margin:0px 0px
                    0px 0.8ex;border-left:1px solid
                    rgb(204,204,204);padding-left:1ex">
                    <div>
                      <div>Hi Michel,<br>
                        <br>
                        Currently, HTTP and SSL are separate
                        measurements. But for SMTP it will probably not
                        work this way... Encryption is not mandatory for
                        SMTP and thus negotiated between client and
                        server every time on port 25. Ports 465 and 587
                        are used for Client Email Submission. You could
                        run some measurements for these ports as well,
                        but for the beginning, i would focus on
                        server2server communication.<br>
                        <br>
                        So here's what i think:<br>
                        <br>
                        What we could measure:<br>
                        - General reachability/availability of the
                        e-mail service<br>
                        - Response time of the remote server: time
                        between connection establish and first SMTP
                        response (220 service ready)<br>
                        - Which enhanced status codes are offered by the
                        server?<br>
                        - Forward/Reverse DNS matching?<br>
                        - SMTP banner matching the DNS name?<br>
                            - if not: what is it?<br>
                        - Does the remote server offer encryption
                        (250-STARTTLS)<br>
                        - Which cipher settings are offered by the
                        server (SSL/TLS Version, Key Exchange
                        Algorithms, Encryption Algorithms, Hashing
                        Algorithms)<br>
                            - alternatively: Which cipher setting has
                        been negotiated between probe and server?<br>
                        - Can we successfully establish a TLS
                        connection?<br>
                        - Certificate Check: is the server-certificate
                        valid and does it match the hostname?<br>
                        - Forced Encryption: MTA-STS available and
                        'enforced' or 'testing' or 'report'?<br>
                        - Forced Authentication: DANE available and
                        check successfull?<br>
                        <br>
                        <br>
                        What we should not do:<br>
                        - send MAIL FROM: command<br>
                        - send RCPT TO: command<br>
                        - send DATA command<br>
                        - measure e-mail delivery/roundtrip time, etc.<br>
                        - Sending e-mails at all<br>
                        <br>
                        The Atlas probe should quit the connection after
                        the data is collected. An actual e-mail should
                        not be send. The target mailserver would count
                        this session as "incomplete" or "aborted" which
                        is totally fine. If someone would want to
                        monitor what happens after a mailserver has
                        successfully accepted a testmail, he should
                        better use a monitoring service/solution. We
                        measure the INTERnet, not what comes after
                        (Intra) :)<br>
                        <br>
                        I don't expect any "spam" problems. Since the
                        Atlas probes wouldn't send e-mails, there's
                        nothing a spamfilter could analyze. The only
                        thing that could theoretically happen, is that
                        the probes ip addresses are flagged as bad by
                        services like Spamhaus etc. and thus be listed
                        on DNSBL/IPBL, but i really don't see a reason
                        why that should happen. There wouldn't be any
                        activity originating from the probes which could
                        be classified as bad. IP addresses are normally
                        only blacklisted, if they send unwanted mails
                        like spam. Also: there are a lot of "check you
                        mailserver" or "check your SSL/TLS" websites.
                        The RIPE Atlas probes would behave the same way.
                        No big deal.<br>
                        <br>
                        Maybe we can think of an "extended" SMTP
                        measurement where RIPE Atlas sends actual
                        e-mails, but that would require (in my opinion),
                        that the person who is creating the measurement
                        somehow provides proof, to be the owner of the
                        target mailserver.<br>
                        <br>
                        <br>
                        BR,<br>
                        Simon<br>
                        <br>
                        <br>
                        On 15.09.22 12:03, Michel Stam wrote:<br>
                      </div>
                      <blockquote type="cite"> Hello Simon,
                        <div><br>
                        </div>
                        <div>Thank you for the suggestion.</div>
                        <div><br>
                        </div>
                        <div>I have a couple of questions to get a
                          better idea:</div>
                        <div>
                          <ul>
                            <li>Can you maybe describe what a SMTP
                              measurement would look like?</li>
                            <ul>
                              <li>Simple EHLO/HELO</li>
                              <li>Sending an email to a designated
                                address (which would then validate that
                                the SMTP server is capable of relaying
                                etc.)</li>
                            </ul>
                            <li>How would DNSBL or other spam prevention
                              techniques fit into this? </li>
                            <li>What would the result be? </li>
                            <ul>
                              <li>Delay until mail received</li>
                              <li>Response time by the actual mail
                                server</li>
                              <li>Using the Received: headers to get a
                                “traceroute” like result.</li>
                            </ul>
                            <li>What about the more uncommon ports such
                              as 565 (SMTP+SSL/TLS) or 587 (mail
                              submission port).</li>
                            <li>How can we prevent this implementation
                              from having RIPE Atlas be identified as a
                              spam bot network?</li>
                          </ul>
                        </div>
                        <div>
                          <div> </div>
                          <div>Best regards,</div>
                          <div><br>
                          </div>
                          <div>Michel</div>
                          <div><br>
                            <blockquote type="cite">
                              <div>On 3 Sep 2022, at 14:48, Simon Brandt
                                via ripe-atlas <<a
                                  href="mailto:ripe-atlas@ripe.net"
                                  target="_blank" rel="noreferrer"
                                  moz-do-not-send="true"
                                  class="moz-txt-link-freetext">ripe-atlas@ripe.net</a>>
                                wrote:</div>
                              <br>
                              <div>
                                <div> Hello,<br>
                                  <br>
                                  i'd like to start a discussion about
                                  having a RIPE Atlas SMTP measurements.<br>
                                  First of all: yes, i know there's a
                                  big obstacle for such a measurement
                                  type. A lot of probes are deployed
                                  using enduser internet-connections
                                  like dsl, cable, etc. with
                                  dynamic/eyeball IP addresses. Those IP
                                  spaces are usually blocked by SMTP
                                  servers as approach to reduce spam
                                  mails. For Example: by using
                                  blocklists like Spamhaus PBL. So a
                                  proper SMTP measurement wouldn't work.<br>
                                  <br>
                                  BUT we could have an easy way for RIPE
                                  Atlas probe hosters to signalize, that
                                  their probe is eligible for SMTP
                                  measurements:<br>
                                  <br>
                                  Step 1: enable "Simple DNS Entry"<br>
                                  Step 2: create a matching reverse DNS
                                  record for the probes IP address<br>
                                  <br>
                                  Everybody who is able so configure a
                                  reverse DNS record for his probes IP
                                  address, is most likely using a
                                  non-dynamic/non-home ip address space
                                  e.g. a datacenter or office network.
                                  If an ISP provides the option for his
                                  customers to configure a reverse DNS
                                  record, it's most likely a
                                  "business-customer" subnet which can
                                  be used to run mailservers. After Step
                                  1+2 are done, the RIPE Atlas platform
                                  would easily be able to verify if
                                  forward-confirmed reverse DNS is
                                  successful, and if so, automatically
                                  enable that probe for SMTP
                                  measurements. Alternatively: probe
                                  hosters choose their own
                                  Forward-confirmed reverse DNS name and
                                  submit it on the RIPE Atlas website.<br>
                                  <br>
                                  Also: if we would have STMP
                                  measurements, forward-confirmed
                                  reverse DNS should be mandatory for
                                  anchors, or is it already?<br>
                                  <br>
                                  Why should we have SMTP measurements?<br>
                                  <br>
                                  Besides general reachability checks,
                                  we could also evaluate SMTP response
                                  codes. But the most important thing
                                  for me is this: the SMTP protocol is
                                  old. Very old. From a security point
                                  of view, it's absolutely outdated.
                                  Most security features have been added
                                  years after the initial RfC. Thus,
                                  those security features are optional.
                                  Mandatory SMTP encryption is not
                                  provided by the SMTP RfC. So both
                                  sides have to signalize, that they are
                                  capable of encryption using the
                                  STARTTLS command. An attacker
                                  (man-in-the-middle) can perform a
                                  downgrade attack by suppressing the
                                  STARTTLS command. So both sides are
                                  forced to fallback and communicate
                                  unencrypted. RIPE Atlas would be a
                                  really good tool to identify such
                                  attacks, by monitor/measure the
                                  (enhanced) status codes of a target.<br>
                                  <br>
                                  But there's more!<br>
                                  I see a two-sided model here. Either
                                  use the RIPE Atlas SMTP measurements
                                  to monitor/measure your own mailserver
                                  by alot of other RIPE probes out
                                  there, OR probe hosters could run SMTP
                                  measurements originating from their
                                  own probe to find out, if their own IP
                                  address is currently blocked by other
                                  mailservers.<br>
                                  <br>
                                  <br>
                                  What do you think?<br>
                                  <br>
                                  <br>
                                  BR,<br>
                                  Simon<br>
                                </div>
                                -- <br>
                                ripe-atlas mailing list<br>
                                <a href="mailto:ripe-atlas@ripe.net"
                                  target="_blank" rel="noreferrer"
                                  moz-do-not-send="true"
                                  class="moz-txt-link-freetext">ripe-atlas@ripe.net</a><br>
                                <a
                                  href="https://mailman.ripe.net/"
                                  target="_blank" rel="noreferrer"
                                  moz-do-not-send="true"
                                  class="moz-txt-link-freetext">https://mailman.ripe.net/</a><br>
                              </div>
                            </blockquote>
                          </div>
                          <br>
                        </div>
                      </blockquote>
                      <br>
                    </div>
                    -- <br>
                    ripe-atlas mailing list<br>
                    <a href="mailto:ripe-atlas@ripe.net" target="_blank"
                      rel="noreferrer" moz-do-not-send="true"
                      class="moz-txt-link-freetext">ripe-atlas@ripe.net</a><br>
                    <a
                      href="https://mailman.ripe.net/"
                      rel="noreferrer noreferrer" target="_blank"
                      moz-do-not-send="true"
                      class="moz-txt-link-freetext">https://mailman.ripe.net/</a><br>
                  </blockquote>
                </div>
              </blockquote>
              <br>
            </div>
          </blockquote>
        </div>
      </blockquote>
      <br>
      <br>
      <fieldset class="moz-mime-attachment-header"></fieldset>
    </blockquote>
    <br>
    <div class="moz-signature">-- <br>
      Paul Theodoropoulos<br>
      <a href="https://www.anastrophe.com" moz-do-not-send="true">anastrophe.com</a></div>
  </body>
</html>