Project

General

Profile

Actions

Regression #11545

open

Primary interface address is not always used when VIPs are present

Added by Kris Phillips 11 months ago. Updated 2 days ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
Interfaces
Target version:
Start date:
02/25/2021
Due date:
% Done:

0%

Estimated time:
Plus Target Version:
Release Notes:
Default
Affected Version:
2.5.0
Affected Architecture:
All

Description

If you have IP Aliases on a WAN interface that a Site to Site IPSec tunnel is riding over and upgrade from 2.4.5p1 to pfSense Plus, you have to go into the WAN interface and hit "Save" and "Apply Configuration" then restart the IPsec service to bring tunnels up post-upgrade. Otherwise IPSec will never connect no matter how many times you cycle the service.

Step by Step:

1. Create IPSec on WAN interface with several IP Aliases
2. Upgrade to 21.02/21.02p1
3. IPSec is broken, so you go into the WAN interface, hit save with no changes, and Apply Changes.
4. Restart IPSec service

Tunnels now work.


Related issues

Related to Bug #3997: get_interface_ip() returns first IP on interface, not necessarily primary IPResolvedChris Buechler11/06/2014

Actions
Has duplicate Bug #12532: Virtual IP problem with OpenVPNDuplicate

Actions
Actions #1

Updated by Steve Wheeler 11 months ago

  • Category changed from IPsec to Interfaces
  • Target version set to Plus-Next

This appears to be a more general issue that can affect IPSec.

In some situations the interface can start to use a VIP IP as the primary address. That causes things running on the interface to fail as they use the wrong address.

I have seen that with an OpenVPN server.

You can see by checking Status > Interfaces.

Resaving the interface corrects the IP allowing services to start.

Actions #2

Updated by Viktor Gurov 11 months ago

Could be the same issue as #5999 (service takes the first IP address on the interface, instead of a non-VIP address)

Actions #3

Updated by Jim Pingle 11 months ago

  • Tracker changed from Bug to Regression
  • Project changed from pfSense Plus to pfSense
  • Subject changed from Upgrading from 2.4.5p1 to 21.02/21.02p1 with IP Aliases on a WAN interface causes IPSec issues to Primary interface address is not always used when VIPs are present
  • Category changed from Interfaces to Interfaces
  • Target version changed from Plus-Next to CE-Next
  • Affected Plus Version deleted (21.02)
  • Affected Version set to 2.5.0

Sounds more like a new variation or regression of #3997

Doubtful that this is specific to Plus, so moving to pfSense.

Actions #4

Updated by Jim Pingle 11 months ago

  • Related to Bug #3997: get_interface_ip() returns first IP on interface, not necessarily primary IP added
Actions #5

Updated by Jim Pingle 11 months ago

  • Target version changed from CE-Next to 2.5.1

Should at least take a stab at this to see if we can come up with a workaround for now.

Actions #6

Updated by Renato Botelho 10 months ago

  • Target version changed from 2.5.1 to CE-Next

Not enough time for 2.5.1

Actions #7

Updated by Kris Phillips 8 months ago

Ran into this again today on a pfSense Plus 21.02.2 upgrade. Had to do the following to fix it:

1. Save the VIP being used by a VTI tunnel without making changes and apply
2. Save the IPSec tunnel using the VTI tunnel
3. Stop and Start the IPSec service

Otherwise the IPSec service would simply say "acquiring job X" and then the "no trap found" for VTI tunnels because of strongswan puking on non-tunnel mode tunnels. No traffic was being generated by the interface at all in a pcap and the IPSec service simply puked before it even start generating traffic.

Easy workaround, but annoying and we should track down what is causing the VIPs to not properly re-tie to the IPSec service.

This affects both OpenVPN and IPSec. May also affect Wireguard depending.

Actions #8

Updated by Steve Wheeler 8 months ago

This only seems to affect VPN tunnels where I assume the interface IP is read directly from the interface causing the problem.

It does not affect outbound NAT for example, where the translation address remains the configured WAN IP and not the incorrect primary interface address.

Actions #9

Updated by Viktor Gurov 8 months ago

might be `ifconfig` bug, like #11594 and #11964

Actions #10

Updated by M Felden 8 months ago

I believe I am seeing this now after upgrading 2.4.5-p1 -> 2.5.1-CE with FRR BGP where FRR is told to use the WAN IPv6 address to establish BGP sessions but it chooses an IPv6 VIP for outbound communication instead. This is rejected by the peer and the session fails. Consequently it brings down all IPv6 because the Virtual IP in question is part of the prefix announced by that BGP session. Removing the IP alias and rebooting will bring us back to normal.

https://forum.netgate.com/topic/164186/virtual-ipv6-addresses-assigned-to-wan-seem-to-take-precence-over-dhcpv6-address-assigned-to-wan-on-outbound-traffic

Actions #11

Updated by Kris Phillips 7 months ago

M Felden wrote:

I believe I am seeing this now after upgrading 2.4.5-p1 -> 2.5.1-CE with FRR BGP where FRR is told to use the WAN IPv6 address to establish BGP sessions but it chooses an IPv6 VIP for outbound communication instead. This is rejected by the peer and the session fails. Consequently it brings down all IPv6 because the Virtual IP in question is part of the prefix announced by that BGP session. Removing the IP alias and rebooting will bring us back to normal.

https://forum.netgate.com/topic/164186/virtual-ipv6-addresses-assigned-to-wan-seem-to-take-precence-over-dhcpv6-address-assigned-to-wan-on-outbound-traffic

Hello M Felden,

Per my previous redmine reply, you only need to resave the VIP and interface. There is no need to remove it, although that is a valid solution as well.

Actions #12

Updated by M Felden 7 months ago

Per my previous redmine reply, you only need to resave the VIP and interface. There is no need to remove it, although that is a valid solution as well.

Hi Kris,

This workaround works but does not persist reboots and certain service restarts seem to break it, too. I can get the box up like this but it is very fragile.

Actions #13

Updated by Denny Page 3 months ago

Still seeing this in 21.05.2... any possibility this will be addressed soon?

Actions #14

Updated by Steve Wheeler 3 months ago

We have been unable to replicate this issue in any sort of repeatable way which makes it almost impossible to dig into.

If anyone has steps to replicate it please add them here.

Actions #15

Updated by Denny Page 3 months ago

I can share info from my install if you like. Unless I disable DHCP6 on the WAN interface, I am currently hitting the issue on every reboot.

Actions #16

Updated by Kris Phillips 3 months ago

Denny Page wrote in #note-15:

I can share info from my install if you like. Unless I disable DHCP6 on the WAN interface, I am currently hitting the issue on every reboot.

Denny,

What version of pfSense are you running right now? Are you saying that VIPs disassociate from their interfaces on every reboot unless you turn off IPv6 on your WAN interface?

Actions #17

Updated by Denny Page 3 months ago

Kris Phillips wrote in #note-16:

What version of pfSense are you running right now?

As noted above, 21.05.2.

Are you saying that VIPs disassociate from their interfaces on every reboot unless you turn off IPv6 on your WAN interface?

No. We're saying that IPsec chooses the wrong address on the WAN interface.

Actions #18

Updated by Jim Pingle 2 months ago

  • Has duplicate Bug #12532: Virtual IP problem with OpenVPN added
Actions #19

Updated by Dan Edwards 2 months ago

Also have the same issue on 21.05.1 on every install in 2 different scenarios. Scenario 1 WAN interface has /29 using the remaining IP's as VIP for NAT etc as IP Alias on the WAN interface so for example 1.1.1.0/29 using 1.1.1.1 as GW 1.1.1.2 as WAN and then 1.1.1.3-6 as VIP IP Alias. Using the wizard for initial configuration everything looks fine and works as expected. Reboot and then in the Dashboard the WAN shows as having the 1st IP Alias as it's address in the Interfaces widget. IPSEC is then also bound to the IP Alias instead of the actual WAN IP address. Workaround is to edit the WAN interface and set IPV6 Configuration type to None. This resolves the issue - WAN interface displays with correct IP address in Interfaces Widget and IPSEC is bound to correct address as well. Scenario 2 WAN Interface is /31 with a /29 routed to it so 1.1.1.2/31 for instance with 2.2.2.0/29 routed to 1.1.1.2 WAN IP Aliases configured for the 2.2.2.x addresses. Again after the initial setup all works but after first reboot WAN displays as 2.2.2.x in the interfaces widget and IPSEC binds to that as well. Go into WAN interface and set IPV6 to None and issue is resolved. It could be any edit on the WAN interface to be fair but have not tried that yet as setting IPV6 to none works - will try next time it happens as we have other installs coming up. With IPV6 set to none it seems to survive a reboot...

Actions #20

Updated by Marcos Mendoza 2 months ago

To clarify, are these new installs, or upgrades? What platform (e.g. AWS)? And yes, try reproducing it and just click Save without changing anything on the interface and let us know if that works and if it survives a reboot.

Actions #21

Updated by Denny Page 2 months ago

I was just bit by this again this morning. Every reboot. Very frustrating. Steve, if you need any information on the configuration, please let me know.

Actions #22

Updated by Dan Edwards 2 months ago

Sorry, new installs on SG2100's and XG7100's, 1 or 2 have been upgraded from 21.05 to 21.05.1 but same issue on all.

Actions #23

Updated by Marcos Mendoza 2 days ago

For anyone that can reproduce this issue, it would be useful to know if this is still occurring in 22.01.

Actions #24

Updated by M Felden 2 days ago

Marcos Mendoza wrote in #note-23:

For anyone that can reproduce this issue, it would be useful to know if this is still occurring in 22.01.

I did not see release notes promising to fix it but will test on 2.6.0-CE-RC this week.

Actions

Also available in: Atom PDF