pfSense

Here's the "same" testing with some more details:

Testing is done in two difference scenarios, where the first has pfsense v23.01 sitting behind a router/modem giving pfsense a WAN IP of 192.168.1.20 in DMZ .

As a reference I start off by doing regular Port Forwarding of the ports known to be required for the games tested (typically 3074 UDP). CoD MW2, MW3, Vanguard and MW2 2022 are the tested games, a bit old and very new.

Reference Test 1 : No UPnP, only port forwarding.
All games using port 3074 show Open NAT as expected. (As an additional test, I moved pfsense out of DMZ in the upstream router, and games then report Moderate NAT instead).
One game, MW2 (2009) show Strict NAT, as I have not figured out the correct ports apparently.

Test 2 : Without removing port forwards, I turn on UPnP without activating STUN.
Starting a game and the System Log (routing) shows last item being
2023-03-08 19:25:14.451357+01:00 miniupnpd 34071 private/reserved address 192.168.3.20 is not suitable for external IP
The UPnP & NAT-PMP rules list is empty, as expected.
Games still show Open NAT as expected (remember, ports are forwarded)

Test 3 : Without removing port forwards, I now turn on STUN for UPnP (using Google server on port 19302).
When a game is started, the rules list now gets populated with the requested ports and reads e.g.: WAN2 udp any 3074 192.168.1.91 3074 DemonwarePortMapping
So clearly UPnP is picking up the request for port 3074 as shown in the list. BUT games fail to connect and eventually show error like : Modern Warfare 3 server not available... etc.
Same thing applies to MW2 which no longer shows Strict NAT, but rather fails to connect at all.
This test renders the same result with or without port forwards active under Firewall > NAT.

In the second scenario I only change the LAN IP on the upstream router so that pfsense has a fake public IP on WAN. Any IP will do as long as it is not considered "private". All other settings in upstream router and pfsense are the same as in scenario 1.

Test 4 : As a reference, again, old style port forwarding still works. Games get Open NAT (except MW2). Same as test 1.

Test 5 : Turning on UPnP, without STUN. There is no longer any log entry about Private IP not being suitable.
When a game is started, the rules list gets populated with the requested ports like e.g.: WAN2 udp any 3074 192.168.1.91 3074 DemonwarePortMapping (exactly like in test #3).
HOWEVER, ALL games, including MW2, now get Open NAT !!

Test 6 : Activating STUN makes no difference. Same result as in Test 5.

So as long as miniupnp "thinks" it has to do with a public IP, things run smoothly, despite the double NAT scenario. Port forwards are not needed at all, and setting Outbound NAT on fully automatic (vs Hybrid), still works 100%.
As soon as miniupnp becomes aware of a Private IP, it will no longer play ball even when it does know the actual public IP on the upstream router. In fact it actually "breaks" a working set up (manual port forwards).

Simplified testing:

Scenario 1.

Upstream router gives pfsense a Private IP in DMZ on WAN.
UPnP settings in pfsense GUI under Services > UPnP & NAT-PMP: Enable, Allow UPnP Port mapping, Allow NAT-PMP Port mapping, External interface WAN, Internal interface LAN, and I activate STUN (using google server) or Override WAN address using the actual Public IP .

Result : in pfsense Status / UPnP & NAT-PMP rules list, the requested port no 3074 UDP is listed together with correct internal IP.
WAN udp any 3074 192.168.1.91 3074 DemonwarePortMapping
None of the games are able to connect at all = worse than Strict NAT

Scenario 2.

Upstream router gives pfsense a fake public IP in DMZ on WAN.
All other settings as in scenario 1: Enable, Allow UPnP Port mapping, Allow NAT-PMP Port mapping, External interface WAN, Internal interface LAN.
However, I do not have to use STUN in order to inform UPnP about the correct external IP. I can use either STUN (google server) OR Override WAN address using the actual Public IP, but doing so makes no difference to the result in this scenario.

Result : in pfsense Status / UPnP & NAT-PMP rules list, the requested port no 3074 UDP is listed together with correct internal IP.
WAN udp any 3074 192.168.1.91 3074 DemonwarePortMapping
All games report Open NAT