[atlas] New V5 Probe with IPv4 Discovery issues

Michel,

First, my apologies for initially misspelling your name! I went ahead and
rebooted the probe while monitoring traffic from it on the LAN and only saw
a single active https session originating from the probe to an edgecast CDN
endpoint in VA over IPv4: https://i.imgur.com/xztP6My.png

I monitored all the DNS queries hitting my server after the reboot and
didn't see any come in for ip-echo.ripe.net either. I could use OPNSense to
capture all outbound DNS traffic from the probe and redirect it to my
server to see if the probe is trying to resolve that host directly
elsewhere, but obviously manipulating the probe's outbound DNS traffic
isn't a viable long term solution for it to function properly. The probe
was generating plenty of other DNS queries hitting my DNS server during
this time though. I also disabled DNS64 as I don't actually have IPv6-only
clients inside my network anymore. Unsurprisingly that did not have an
effect on the IPv4 issue given I'm not seeing DNS requests for the ip-echo
test. The DNS requests made to my DNS server since the probe was reset (all
requests after 11:41) can be seen here: https://i.imgur.com/AlLKeCk.png

OPNSense is fully up to date on release 22.7.8. I'm at a bit of a loss as
to what is going on as well. Is there a way to pull logs or diagnostics
from the probe itself to see what it thinks is going on?

-Nate

On Wed, Nov 30, 2022 at 10:13 AM Michel Stam <mstam at ripe.net> wrote:

> Hello Nate,
>
> Can you check that there are active v4 TCP sessions, namely a port 443/TCP
> session? That should be there for the probe to be able to talk to the
> infrastructure.
>
> Depending on the measurement the probe may directly contact name servers,
> correct, but it may also use your provided name servers.
>
> I’m not sure how often the echo queries happen, but I expect during
> initial connect, not continuously. A simple reboot would prove that.
>
> I don’t see a particular problem in your description as yet, I’ve seen the
> probes happily running on v4 / v6 dual stack networks (no DNS64 though)
> without any problems. Not sure what makes it different here, is there maybe
> a more recent version of OPNSense available
>
> It’s important that the probe is not restricted from accessing the
> Internet, because this is the nature of the measurements it does. It will
> also use old-style DNS, not only DNS over TLS, not sure if this is a
> problem.
>
> Keep me posted.
>
> Cheers,
>
> Michel
>
>
> On 30 Nov 2022, at 17:00, Nate Weibley <nweibley at gmail.com> wrote:
>
> Michael,
>
> Thank you for your troubleshooting guidance. I do not see any evidence of
> OPNSense blocking the IPv4 traffic originating from the probe; to the
> contrary I see IPv4 UDP and ICMP traffic passing through the LAN interface
> via the default allow rule (I have the probe on my main LAN to establish
> everything is working and will move it to my isolated IOT VLAN once
> comfortable). Many other IPv4 devices also connected to the LAN network
> function normally over IPv4.
>
> Here's a screenshot of all the probe IPv4 traffic I see, as well as all of
> the traffic which does not match a pass rule on all non-WAN interfaces in
> my network: https://imgur.com/a/GCMR9sz
>
> Additionally I do not see any recent DNS queries for ip-echo.ripe.net in
> my DNS resolver logs, though I suspect the probe may be directly querying
> other resolvers directly for those requests (I do see plenty of other DNS
> traffic from the probe on my local DNS server though). I was
> initially suspicious that having DNS64 support enabled in my unbound
> resolver may be to blame, but since I don't see requests hitting it I'm not
> confident that is the case. My upstream DNS servers (I run Adguard Home
> locally for my LAN clients, though the probe has been set to bypass all
> filter rules) are my local OPNSense unbound instance and DNS-over-TLS to
> 1.1.1.1 and 8.8.8.8.
>
> I will keep poking around to see if I can uncover anything that may be
> getting blocked or rerouted from the probe.
>
> -Nate
>
> On Wed, Nov 30, 2022 at 5:18 AM Michel Stam <mstam at ripe.net> wrote:
>
>> Hello Nate,
>>
>> Sorry to hear you’ve been having problems with the V5 probe.
>>
>> It seems that your probe is not able to contact the IP echo service at
>> ip-echo.ripe.net to derive its IP address. Because of this, IPv4 is
>> presumed to be non-functional and the probe will not execute IPv4
>> measurements.
>>
>> Can you take a good look at the OPNSense configuration to see if any v4
>> traffic going towards the internet is being filtered?
>>
>> Regards,
>>
>> Michel
>>
>> On 28 Nov 2022, at 16:31, Nate Weibley <nweibley at gmail.com> wrote:
>>
>> I received a shiny new V5 probe via post on Saturday and got it up and
>> running yesterday on my dual-stack network.  The probe in question is:
>> https://atlas.ripe.net/probes/62490/
>>
>> It has been operational for about 18 hours and quickly indicated that
>> IPv6 connectivity was functional, but it has only momentarily indicated
>> IPv4 connectivity was established. I can see IPv4 ICMP and UDP traffic
>> originating from the probe passing through my opnsense router on the IPv4
>> WAN interface so I know IPv4 traffic is flowing. I also see the correct
>> IPv4 DHCP lease info for my LAN on the probe's network status page.
>>
>> Probe address discovery does not show a valid IPv4 connection address or
>> IP Echo Service listed, only the Local IP of my probe. I've tried power
>> cycling the probe but it doesn't seem to impact this issue.
>>
>> Are there any other troubleshooting steps I should try?
>> --
>> ripe-atlas mailing list
>> ripe-atlas at ripe.net
>> https://mailman.ripe.net/
>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: </ripe/mail/archives/ripe-atlas/attachments/20221130/8cd11aa2/attachment.html>