Project

General

Profile

Actions
Copy link

Regression #11995

closed

UPnP/NAT-PMP not functioning on 32-bit ARM

Added by Jim Pingle over 3 years ago. Updated about 3 years ago.

Status:
Closed
Priority:
High
Assignee:
Renato Botelho
Category:
UPnP/NAT-PMP
Target version:
21.05.1
Start date:
06/04/2021
Due date:
% Done:

0%

Estimated time:
Release Notes:
Default
Affected Plus Version:
21.05
Affected Architecture:

Description

UPnP is not functional on 32-bit ARM systems (SG-3100, SG-1000) running pfSense Plus 21.05. When a client attempts to map a port, an error is logged by miniupnpd and no mapping is created. The same configuration works fine on 64-bit ARM (SG-1100) and amd64.

Jun 4 09:01:15     miniupnpd     49647     HTTP listening on port 2189
Jun 4 09:01:15     miniupnpd     49647     HTTP IPv6 address given to control points : [2001:db8:1:eeb0:6a9e:19ff:fe87:be86]
Jun 4 09:01:15     miniupnpd     49647     setsockopt(udp, IPV6_RECVPKTINFO): Invalid argument
Jun 4 09:01:15     miniupnpd     49647     Listening for NAT-PMP/PCP traffic on port 5351
Jun 4 09:12:35     miniupnpd     49647     ioctl(dev, DIOCGETADDRS, ...): Device busy
Jun 4 09:12:35     miniupnpd     49647     ioctl(dev, DIOCGETADDRS, ...): Device busy
Jun 4 09:12:35     miniupnpd     49647     ioctl(dev, DIOCGETADDRS, ...): Device busy
Jun 4 09:12:35     miniupnpd     49647     ioctl(dev, DIOCGETADDRS, ...): Device busy
Jun 4 09:12:35     miniupnpd     49647     ioctl(dev, DIOCGETADDRS, ...): Device busy
Jun 4 09:12:35     miniupnpd     49647     ioctl(dev, DIOCGETADDRS, ...): Device busy
Jun 4 09:12:35     miniupnpd     49647     ioctl(dev, DIOCGETADDRS, ...): Device busy
Jun 4 09:12:35     miniupnpd     49647     ioctl(dev, DIOCGETADDRS, ...): Device busy
Jun 4 09:12:35     miniupnpd     49647     ioctl(dev, DIOCGETADDRS, ...): Device busy
Jun 4 09:12:35     miniupnpd     49647     ioctl(dev, DIOCGETADDRS, ...): Device busy

In each case the configuration is simple - the service is enabled and has both UPnP and NAT-PMP enabled, plus I have set a WAN Override address to work around the fact that the lab systems have private WANs.

Files

Download all files
kdump-out.txt (153 KB) kdump-out.txt Jim Pingle, 07/06/2021 01:14 PM
miniupnp-debug-amd64.txt (14 KB) miniupnp-debug-amd64.txt Jim Pingle, 07/06/2021 04:03 PM
miniupnp-debug-sg3100.txt (34.9 KB) miniupnp-debug-sg3100.txt Jim Pingle, 07/06/2021 04:03 PM
Actions
Copy link
 #1

Updated by Jim Pingle over 3 years ago

This still happens on current 21.09 snapshots (21.09.a.20210706.0500):

Jul 6 12:56:31     miniupnpd     68462     ioctl(dev, DIOCGETADDRS, ...): Device busy
Jul 6 12:56:31     miniupnpd     68462     ioctl(dev, DIOCGETADDRS, ...): Device busy
Jul 6 12:56:31     miniupnpd     68462     ioctl(dev, DIOCGETADDRS, ...): Device busy
Actions
Copy link
 #2

Updated by Jim Pingle over 3 years ago

It's also noteworthy that it IS adding some rules, but they are block return firewall rules and not the nat and rdr rules we expect to see.

miniupnpd rules/nat contents:
block return quick on ! mvneta2 from any to any port = 55784 label "NAT-PMP 55784 tcp" rtable 0
block return quick on ! mvneta2 from any to any port = 4433 label "NAT-PMP 4433 tcp" rtable 0
block return quick on ! mvneta2 from any to any port = 55784 label "NAT-PMP 55784 udp" rtable 0
block return-icmp(0, 0) quick on ! mvneta2 from ?/32 port = 55784 to any label "NAT-PMP 55784 udp" rtable 0
block return quick on ! mvneta2 from any to any port = 55784 label "Deluge 1.3.15 at 10.21.0.10:55784" rtable 0
block return quick on ! mvneta2 from any to any port = 4433 label "Deluge 1.3.15 at 10.21.0.10:4433" rtable 0
block return quick on ! mvneta2 from any to any port = 55784 label "Deluge 1.3.15 at 10.21.0.10:55784" rtable 0
block return-icmp(0, 0) quick on ! mvneta2 from ?/32 port = 55784 to any label "Deluge 1.3.15 at 10.21.0.10:55784" rtable 0

After some debugging with Mateusz, he says: "the error stems from NULL returned by pf_get_kpool"

Loaded a special kernel with some printfs around the error and saw:

debug_pf_get_kpool: pid 90992 line 404
debug_pf_get_kpool: pid 90992 line 446
pfioctl: pid 90992 line 3161

That was repeated once for each error in the log.

Ran a krace on the process as well:

ktrace -p 90992
<start a UPnP-aware client and wait for the errors in the log>
ktrace -C
kdump > out.txt

Results of that are attached.

Actions
Copy link
 #3

Updated by Jim Pingle over 3 years ago

It doesn't appear to be due to a change in the ports, as 21.02.2 works and has miniupnpd-2.2.1,1 while 21.05 fails and has the exact same version, miniupnpd-2.2.1,1

Both contain the same OPTIONS:

Options        :
    CHECK_PORTINUSE: on
    IPV6           : on
    LEASEFILE      : off
    PF_FILTER_RULES: on
    UPNP_IGDV2     : off
    UPNP_STRICT    : off

I attached the output of one working and one non-working UPnP session from the same client. The working one against amd64, failing one against SG-3100, both on the latest 21.09 snapshot. There are quite a few more errors on the SG-3100 file, though none point to anything obvious to me. It appears to want to add the rule, but it fails, and then has errors resulting from that failure.

Actions
Copy link
 #4

Updated by Jim Pingle over 3 years ago

  • Assignee set to Renato Botelho

It looks like this may be from a change in the FreeBSD kernel between versions that required a new build of miniupnpd, but the version number of miniupnpd didn't increase so it didn't get reinstalled.

You should be able to fix this with:

killall -9 miniupnpd
pkg upgrade -fy miniupnpd

And then click Save on the miniupnpd settings.

Actions
Copy link
 #5

Updated by Renato Botelho over 3 years ago

  • Status changed from New to Feedback

I've bumped miniupnpd package to `2.2.1_1,1` on 2.6.0/2.5.2 CE and 21.09/21.05 Plus

Actions
Copy link
 #6

Updated by Jim Pingle about 3 years ago

  • Status changed from Feedback to Closed
  • Target version changed from 64 to 21.05.1

This was fixed before 21.05.1

Actions
Copy link

Also available in: Atom PDF