Regression #11995
closed
UPnP/NAT-PMP not functioning on 32-bit ARM
0%
Description
UPnP is not functional on 32-bit ARM systems (SG-3100, SG-1000) running pfSense Plus 21.05. When a client attempts to map a port, an error is logged by miniupnpd and no mapping is created. The same configuration works fine on 64-bit ARM (SG-1100) and amd64.
Jun 4 09:01:15 miniupnpd 49647 HTTP listening on port 2189 Jun 4 09:01:15 miniupnpd 49647 HTTP IPv6 address given to control points : [2001:db8:1:eeb0:6a9e:19ff:fe87:be86] Jun 4 09:01:15 miniupnpd 49647 setsockopt(udp, IPV6_RECVPKTINFO): Invalid argument Jun 4 09:01:15 miniupnpd 49647 Listening for NAT-PMP/PCP traffic on port 5351 Jun 4 09:12:35 miniupnpd 49647 ioctl(dev, DIOCGETADDRS, ...): Device busy Jun 4 09:12:35 miniupnpd 49647 ioctl(dev, DIOCGETADDRS, ...): Device busy Jun 4 09:12:35 miniupnpd 49647 ioctl(dev, DIOCGETADDRS, ...): Device busy Jun 4 09:12:35 miniupnpd 49647 ioctl(dev, DIOCGETADDRS, ...): Device busy Jun 4 09:12:35 miniupnpd 49647 ioctl(dev, DIOCGETADDRS, ...): Device busy Jun 4 09:12:35 miniupnpd 49647 ioctl(dev, DIOCGETADDRS, ...): Device busy Jun 4 09:12:35 miniupnpd 49647 ioctl(dev, DIOCGETADDRS, ...): Device busy Jun 4 09:12:35 miniupnpd 49647 ioctl(dev, DIOCGETADDRS, ...): Device busy Jun 4 09:12:35 miniupnpd 49647 ioctl(dev, DIOCGETADDRS, ...): Device busy Jun 4 09:12:35 miniupnpd 49647 ioctl(dev, DIOCGETADDRS, ...): Device busy
In each case the configuration is simple - the service is enabled and has both UPnP and NAT-PMP enabled, plus I have set a WAN Override address to work around the fact that the lab systems have private WANs.
Files
Updated by Jim Pingle over 3 years ago
This still happens on current 21.09 snapshots (21.09.a.20210706.0500):
Jul 6 12:56:31 miniupnpd 68462 ioctl(dev, DIOCGETADDRS, ...): Device busy Jul 6 12:56:31 miniupnpd 68462 ioctl(dev, DIOCGETADDRS, ...): Device busy Jul 6 12:56:31 miniupnpd 68462 ioctl(dev, DIOCGETADDRS, ...): Device busy
Updated by Jim Pingle over 3 years ago
- File kdump-out.txt kdump-out.txt added
It's also noteworthy that it IS adding some rules, but they are block return firewall rules and not the nat and rdr rules we expect to see.
miniupnpd rules/nat contents: block return quick on ! mvneta2 from any to any port = 55784 label "NAT-PMP 55784 tcp" rtable 0 block return quick on ! mvneta2 from any to any port = 4433 label "NAT-PMP 4433 tcp" rtable 0 block return quick on ! mvneta2 from any to any port = 55784 label "NAT-PMP 55784 udp" rtable 0 block return-icmp(0, 0) quick on ! mvneta2 from ?/32 port = 55784 to any label "NAT-PMP 55784 udp" rtable 0 block return quick on ! mvneta2 from any to any port = 55784 label "Deluge 1.3.15 at 10.21.0.10:55784" rtable 0 block return quick on ! mvneta2 from any to any port = 4433 label "Deluge 1.3.15 at 10.21.0.10:4433" rtable 0 block return quick on ! mvneta2 from any to any port = 55784 label "Deluge 1.3.15 at 10.21.0.10:55784" rtable 0 block return-icmp(0, 0) quick on ! mvneta2 from ?/32 port = 55784 to any label "Deluge 1.3.15 at 10.21.0.10:55784" rtable 0
After some debugging with Mateusz, he says: "the error stems from NULL returned by pf_get_kpool"
Loaded a special kernel with some printfs around the error and saw:
debug_pf_get_kpool: pid 90992 line 404 debug_pf_get_kpool: pid 90992 line 446 pfioctl: pid 90992 line 3161
That was repeated once for each error in the log.
Ran a krace on the process as well:
ktrace -p 90992 <start a UPnP-aware client and wait for the errors in the log> ktrace -C kdump > out.txt
Results of that are attached.
Updated by Jim Pingle over 3 years ago
- File miniupnp-debug-amd64.txt miniupnp-debug-amd64.txt added
- File miniupnp-debug-sg3100.txt miniupnp-debug-sg3100.txt added
It doesn't appear to be due to a change in the ports, as 21.02.2 works and has miniupnpd-2.2.1,1 while 21.05 fails and has the exact same version, miniupnpd-2.2.1,1
Both contain the same OPTIONS:
Options :
CHECK_PORTINUSE: on
IPV6 : on
LEASEFILE : off
PF_FILTER_RULES: on
UPNP_IGDV2 : off
UPNP_STRICT : off
I attached the output of one working and one non-working UPnP session from the same client. The working one against amd64, failing one against SG-3100, both on the latest 21.09 snapshot. There are quite a few more errors on the SG-3100 file, though none point to anything obvious to me. It appears to want to add the rule, but it fails, and then has errors resulting from that failure.
Updated by Jim Pingle over 3 years ago
- Assignee set to Renato Botelho
It looks like this may be from a change in the FreeBSD kernel between versions that required a new build of miniupnpd, but the version number of miniupnpd didn't increase so it didn't get reinstalled.
You should be able to fix this with:
killall -9 miniupnpd pkg upgrade -fy miniupnpd
And then click Save on the miniupnpd settings.
Updated by Renato Botelho over 3 years ago
- Status changed from New to Feedback
I've bumped miniupnpd package to `2.2.1_1,1` on 2.6.0/2.5.2 CE and 21.09/21.05 Plus
Updated by Jim Pingle about 3 years ago
- Status changed from Feedback to Closed
- Target version changed from 64 to 21.05.1
This was fixed before 21.05.1