Regression #15430
open
Interface-bound state policy does not handle IPsec VTI traffic as expected when filtering on enc0
0%
Description
https://forum.netgate.com/topic/187632/24-03-frr-has-flapping-bgp-neighbors/3
In my set up there are two VPN types where dynamic routing is occurring. Wireguard and IPsec.
IPsec has been unstable since the upgrade to 24.03.
Digging into the issue it is related to the state policy change. Adding a specific rule for BGP from neighbor to my pfsense firewall on port 179 and changing the state policy to Floating allowed BGP to remain UP without the constant flapping.
Unsure why there is a difference between the two VPN types to pf.
Redmine for tracking.
No need to troubleshoot as problem is resolved. Noting here for additional follow up if required and corner case testing by developers if deemed necessary.
Related issues
Updated by Jim Pingle 10 days ago
- Project changed from pfSense Plus to pfSense
- Category changed from Routing to Routing
What type of IPsec VPN, policy-based or VTI? Since you mention BGP, I'm guessing VTI, but it needs to be confirmed.
The if-bound state policy is much less forgiving when it comes to things like asymmetric routing, but it is possible that isn't the case and it's something in how the states are handled there.
Updated by Jim Pingle 10 days ago
- Has duplicate Bug #15431: Interface Bound Firewall State Policy Breaks IPsec VTI added
Updated by Mike Moore 10 days ago
VTI mode for IPsec.
To reiterate, Wireguard VPN w/ BGP saw no issues.
Updated by Jim Pingle 10 days ago
- Subject changed from state policy changes inconsistent per VPN to Interface-bound state policy does not handle IPsec VTI traffic as expected
IPsec is fundamentally different in how it's handled compared to things like WireGuard/OpenVPN/OpenVPN+DCO. IPsec can be handled in certain ways that make the traffic appear to enter and exit on different interfaces (e.g. ipsecX vs enc0), which naturally doesn't play well with if-bound states but would pass OK with floating states.
What do you have set for the "IPsec Filter Mode" under VPN > IPsec, on the Advanced Settings tab?
Updated by Mike Moore 10 days ago
IPsec Filter Mode set to 'Filter IPsec Tunnel, Transport and VTI on IPsec tab'
Updated by Jim Pingle 10 days ago
- Subject changed from Interface-bound state policy does not handle IPsec VTI traffic as expected to Interface-bound state policy does not handle IPsec VTI traffic as expected when filtering on enc0
- Assignee set to Kristof Provost
- Target version set to 2.8.0
- Plus Target Version set to 24.07
If you do not have any tunnel mode IPsec (no site to site tunnel mode P2s, no mobile IPsec) you could change the filter mode to the other option and then add rules on a tab for the assigned IPsec interface(s) which would work better for this.
However if you have any policy-based/tunnel mode tunnels or mobile IPsec then you have to keep it on the setting you are using, and will likely need to use the floating state policy on IPsec rules.
I'll keep this open for the time being to see if there is anything we can do about this internally but given how IPsec works in that mode it may not be compatible with if-bound states.
Updated by Jim Pingle 10 days ago
- Related to Feature #11395: Option to switch IPsec filtering modes to choose between ``enc`` and ``if_ipsec`` filtering added
Updated by Jim Pingle 10 days ago
- Related to Bug #8686: IPsec VTI: Assigned interface firewall rules are never parsed added
Updated by Mike Moore 10 days ago
I have made the firewall a VTI/Routed IPsec gateway moving forward.
Considering this drawback is noted in the documentation I don't think its a huge deal but I would mention that without putting 2 and 2 together there is no obvious way to know that the default behavior of the state policy would break FRR neighbor peering. Either the peering doesn't come up OR its flapping and its not obvious what the problem is. I am not sure whats the best way to handle that short of giving a blurb somewhere in the FRR menu or in the VPN menu.
Updated by Jim Pingle 10 days ago
I added notes about this to the docs about state policy in general (and in the release notes): https://docs.netgate.com/pfsense/en/latest/config/advanced-firewall-nat.html#interface-bound-states
It's not specific to FRR that just happens to be how you encountered it.
It would be great to see this addressed in the OS and/or PF if possible though so we'll keep this open for the time being unless further analysis reveals it's not feasible to fix directly.
If we can't fix it in the OS or PF we could also change the backend rule generation to use floating policy for tunneled IPsec traffic in/out, we had to make a similar workaround for that in another edge case.