pfSense

Could be the same issue as #5999 (service takes the first IP address on the interface, instead of a non-VIP address)

Ran into this again today on a pfSense Plus 21.02.2 upgrade. Had to do the following to fix it:

1. Save the VIP being used by a VTI tunnel without making changes and apply
2. Save the IPSec tunnel using the VTI tunnel
3. Stop and Start the IPSec service

Otherwise the IPSec service would simply say "acquiring job X" and then the "no trap found" for VTI tunnels because of strongswan puking on non-tunnel mode tunnels. No traffic was being generated by the interface at all in a pcap and the IPSec service simply puked before it even start generating traffic.

Easy workaround, but annoying and we should track down what is causing the VIPs to not properly re-tie to the IPSec service.

This affects both OpenVPN and IPSec. May also affect Wireguard depending.

This only seems to affect VPN tunnels where I assume the interface IP is read directly from the interface causing the problem.

It does not affect outbound NAT for example, where the translation address remains the configured WAN IP and not the incorrect primary interface address.

might be `ifconfig` bug, like #11594 and #11964

I believe I am seeing this now after upgrading 2.4.5-p1 -> 2.5.1-CE with FRR BGP where FRR is told to use the WAN IPv6 address to establish BGP sessions but it chooses an IPv6 VIP for outbound communication instead. This is rejected by the peer and the session fails. Consequently it brings down all IPv6 because the Virtual IP in question is part of the prefix announced by that BGP session. Removing the IP alias and rebooting will bring us back to normal.

https://forum.netgate.com/topic/164186/virtual-ipv6-addresses-assigned-to-wan-seem-to-take-precence-over-dhcpv6-address-assigned-to-wan-on-outbound-traffic

M Felden wrote:

I believe I am seeing this now after upgrading 2.4.5-p1 -> 2.5.1-CE with FRR BGP where FRR is told to use the WAN IPv6 address to establish BGP sessions but it chooses an IPv6 VIP for outbound communication instead. This is rejected by the peer and the session fails. Consequently it brings down all IPv6 because the Virtual IP in question is part of the prefix announced by that BGP session. Removing the IP alias and rebooting will bring us back to normal.

https://forum.netgate.com/topic/164186/virtual-ipv6-addresses-assigned-to-wan-seem-to-take-precence-over-dhcpv6-address-assigned-to-wan-on-outbound-traffic

Hello M Felden,

Per my previous redmine reply, you only need to resave the VIP and interface. There is no need to remove it, although that is a valid solution as well.

Per my previous redmine reply, you only need to resave the VIP and interface. There is no need to remove it, although that is a valid solution as well.

Hi Kris,

This workaround works but does not persist reboots and certain service restarts seem to break it, too. I can get the box up like this but it is very fragile.

Still seeing this in 21.05.2... any possibility this will be addressed soon?

We have been unable to replicate this issue in any sort of repeatable way which makes it almost impossible to dig into.

If anyone has steps to replicate it please add them here.

I can share info from my install if you like. Unless I disable DHCP6 on the WAN interface, I am currently hitting the issue on every reboot.

Denny Page wrote in #note-15:

I can share info from my install if you like. Unless I disable DHCP6 on the WAN interface, I am currently hitting the issue on every reboot.

Denny,

What version of pfSense are you running right now? Are you saying that VIPs disassociate from their interfaces on every reboot unless you turn off IPv6 on your WAN interface?

Kris Phillips wrote in #note-16:

What version of pfSense are you running right now?

As noted above, 21.05.2.

Are you saying that VIPs disassociate from their interfaces on every reboot unless you turn off IPv6 on your WAN interface?

No. We're saying that IPsec chooses the wrong address on the WAN interface.

Also have the same issue on 21.05.1 on every install in 2 different scenarios. Scenario 1 WAN interface has /29 using the remaining IP's as VIP for NAT etc as IP Alias on the WAN interface so for example 1.1.1.0/29 using 1.1.1.1 as GW 1.1.1.2 as WAN and then 1.1.1.3-6 as VIP IP Alias. Using the wizard for initial configuration everything looks fine and works as expected. Reboot and then in the Dashboard the WAN shows as having the 1st IP Alias as it's address in the Interfaces widget. IPSEC is then also bound to the IP Alias instead of the actual WAN IP address. Workaround is to edit the WAN interface and set IPV6 Configuration type to None. This resolves the issue - WAN interface displays with correct IP address in Interfaces Widget and IPSEC is bound to correct address as well. Scenario 2 WAN Interface is /31 with a /29 routed to it so 1.1.1.2/31 for instance with 2.2.2.0/29 routed to 1.1.1.2 WAN IP Aliases configured for the 2.2.2.x addresses. Again after the initial setup all works but after first reboot WAN displays as 2.2.2.x in the interfaces widget and IPSEC binds to that as well. Go into WAN interface and set IPV6 to None and issue is resolved. It could be any edit on the WAN interface to be fair but have not tried that yet as setting IPV6 to none works - will try next time it happens as we have other installs coming up. With IPV6 set to none it seems to survive a reboot...

To clarify, are these new installs, or upgrades? What platform (e.g. AWS)? And yes, try reproducing it and just click Save without changing anything on the interface and let us know if that works and if it survives a reboot.

I was just bit by this again this morning. Every reboot. Very frustrating. Steve, if you need any information on the configuration, please let me know.

Sorry, new installs on SG2100's and XG7100's, 1 or 2 have been upgraded from 21.05 to 21.05.1 but same issue on all.

For anyone that can reproduce this issue, it would be useful to know if this is still occurring in 22.01.

Marcos Mendoza wrote in #note-23:

For anyone that can reproduce this issue, it would be useful to know if this is still occurring in 22.01.

I did not see release notes promising to fix it but will test on 2.6.0-CE-RC this week.