Bug #15625
closed
IPv6 prefix rotation by ISP causes complete service disruption
0%
Description
Whats the issue?
If my ISP hands out a new GUA, the WAN interface only gets an additional new SLAAC or DHCPv6 interface address.
The old address persists on the wan interface, but tagged deprecated. (So the kernel gets the message, but it ends there)
All tracked interfaces retain their old address/prefix. Which is then usually also unrouted by upstream. Furthermore RA still continues to announce the old prefix'es as valid, downstream. This leaves everything without internet connectivity resulting in a complete service disruption.
The pfsense GUI continues to display the deprecated IPv6 under interface status on the WAN interface, even if a new valid address is present on the interface itself.
The DynDNS updater also only reads the deprecated address, resulting in not updating dyn accounts on address change.
This is related to the previous mentioned point and shares a common culprit in https://github.com/pfsense/FreeBSD-ports/blob/devel/devel/php-pfSense-module/files/pfSense.c#L1862 (pfSense_get_interface_addresses)
What do I expext?
If a prefix change occurs pfSense shall * not continue display deprecated prefixes, but show only valid GUA instead if present (ususally the case) * update all related tracked interfaces and renew delegated prefixes * reload firewall rules in order for proposed patch https://github.com/pfsense/pfsense/commit/3c0de6d7905a9f0e1c972a25177e7c642c83de0a.patch to work. I also have a similar patch for NPt and aliases still in mind. * make sure that related services react accordingly to fulfill their basic purpose
I guess prioritizing valid GUA's over deprecated ones in pfSense_get_interface_addresses should allready make a huge improvement.
Related issues
Updated by Marcos M 9 months ago
- Status changed from New to Feedback
- Release Notes changed from Default to Force Exclusion
I don't know if there's any code that still uses pfSense_get_interface_addresses() - most if not all has been updated to use pfSense_get_ifaddrs() instead via the PHP function get_interface_addresses(). AFAIK the related issues have been resolved; details can be found on #11570.
Updated by Jan-Jonas Sämann 9 months ago
Thank You for your answer. I don't think the mentioned issue #11570 is related this which just happened before filing this on a pretty decent 27.7 version. Also I do not have any link state events, which is completely different.
So generally speaking, it is not relevant how the function name is or was, as long as the function in charge of determining the current GUA on an interface does not return the deprecated (from multiple) address as primary address. I mean the function in charge for scraping the addresses shown on the pfSense Dashboard or used by the dynDNS script. So as a result, this breaks stuff down hill.
Where this shares a very far relation to the mentioned issue #11570 is, that both issues take down internet access for connected downstream networks. My firewall itself applied new ipv6 routes and was reachable through the newly provisioned ipv6 GUA. Also #11570 does not seem to address tracked interface renewal on prefix invalidation. As long as pfSense does not support NAT66, there is some work to do to get basic v6 operational in the field.
Updated by Manuel Gayer 3 months ago
Same here, just added my note to the wrong ticket: #12947#note-24 a week ago...
I can confirm and reproduce this on 24.11 / 2.7.2.
The same thing is described in #15906 and #15900, were I added my workaround #15906#note-1 inspired by #12947#note-16
Updated by Jan-Jonas Sämann 3 months ago
Marcos M wrote in #note-1:
I don't know if there's any code that still uses
pfSense_get_interface_addresses()- most if not all has been updated to usepfSense_get_ifaddrs()instead via the PHP functionget_interface_addresses(). AFAIK the related issues have been resolved; details can be found on #11570.
You are right. Either should pfSense_get_ifaddrs() return for instance a "deprecated" attribute to carry the address state in pfSense_get_ifaddrs()["addrs6"][].deprecated or not return deprecated addresses at all.
Another idea would be to reverse the addrs6 list in get_interface_addresses(). Not sure what determines the order of addresses returned by getifaddrs() and if it is just pure luck to always get the wrong address.
Turns out, the code in pfSense.c seems to sort the addresses alphabetically which should return the lowest address first. This is indeed usually the oldest when providers rotate them. But still random, just a theoretical higher probability that the lower address was the older address. The php code then iterates over the addrs6 list and selects the first GUA it finds.
Still not a solution but something we can patch quickly in php code.
Updated by Marcos M 3 months ago
- Is duplicate of Bug #12947: Old IPv6 addresses may continue to be used after DHCP or RA changes added
Updated by Jan-Jonas Sämann 3 months ago
I implemented a deprecated address check and will observe the behavior in the next few days.
Basically this patch provides a new php function is_v6deprecated which scrapes ifconfig, searching for a given inet6 address with a deprecated flag. This then only breaks the primary address searching loop only if the selected GUA address is also not deprecated.
WIP, use with caution. Also it is still unknown if this finally resolves instant propagation on tracked interfaces. At least this should bring us a little step forward in terms of visibility and other features requiring a routeable GUA.