Bug #16123
open
Advertisements from a GUA are ignored
0%
Description
ISP: Gigaclear (UK)
Description:
After exactly 5 minutes v6 connectivity will die. Prefixes are still present, but no v6 traffic flows.
Gigaclear state that their head end is sending NS requests every few seconds with no neighbour advertisements being received on the interface that has the /48 prefix allocated to it. Neighbour Discovery fails because of this.
When ND breaks the layer2 detection mechanism signals to the subscriber management platform that v6 session is dead, severs the v6 binding, breaking v6 connectivity.
Observations
- Please see attached packet captures
- A capture shows a neighbourhood solicitation request from pfSense to a link-local address of a Juniper switch
- The Juniper switch instead responds from its Global Unique Address.
- pfSense ignores the request as it is being seen from a GUA.
- I have no explanation why the Juniper switch is sending Link-Local messages from its GUA.
Why is pfSense ignoring these advertisements? A temporary workaround is to add a v6 static route to the first hop, but this will time out after a (currently unknown) period of time.
Files
Updated by quiet lion 13 days ago
NB: Adding a static route to the first v6 hop allows the NDP to be inserted:
ndp -a
2a02:fb8::11 4e:6d:58:87:10:17 igb0 8h17m28s S R
BUT, this will fail when the end user is migrated to another BNG.
Updated by Marcos M 13 days ago
- Status changed from New to Incomplete
- Priority changed from Very High to Normal
NA's are not necessarily restricted to LL addresses and the default ruleset allows this. I'm unable to reproduce the issue between two pfSense devices using version 25.03. Presumably 64:62:66:21:6b:75 is pfSense on those pcaps; I don't see any GUA-sourced neighbor solicitations for IPv6 addresses belonging to that MAC address.
It's best to troubleshoot this further before opening a bug report. Contact TAC for assistance or follow up on the forum.
Updated by quiet lion 12 days ago
- File packetcapture-igb1-20250404025127-LAN.pcap packetcapture-igb1-20250404025127-LAN.pcap added
- File packetcapture-igb0-20250404025028.pcap packetcapture-igb0-20250404025028.pcap added
Appreciate the reply, Marcos.
NA's are not necessarily restricted to LL addresses and the default ruleset allows this. I'm unable to reproduce the issue between two pfSense devices using version 25.03.
There's a whole thread on this in another forum. Most relevant post is here: https://www.ispreview.co.uk/talk/threads/anyone-noticed-if-gigaclear-have-deployed-ipv6.43168/post-412456
There are multiple people on Gigaclear all reporting the same exact issue on pfSense. Linux based firewalls work just fine.
Presumably
64:62:66:21:6b:75is pfSense on those pcaps;
Correct. I've reuploaded new captures on the WAN & LAN ifaces. There is no v6 packets flowing when I performed the capture.
I don't see any GUA-sourced neighbor solicitations for IPv6 addresses belonging to that MAC address.Gigaclear sent me these logs from their side:
In IP6 fe80::6662:66ff:fe21:6b74 > fe80::4e6d:58ff:fe87:1017: ICMP6, neighbor solicitation, who has fe80::4e6d:58ff:fe87:1017, length 32
Out IP6 2a02:fb8::11 > fe80::6662:66ff:fe21:6b74: ICMP6, neighbor advertisement, tgt is fe80::4e6d:58ff:fe87:1017, length 24
In IP6 fe80::6662:66ff:fe21:6b74 > fe80::4e6d:58ff:fe87:1017: ICMP6, neighbor solicitation, who has fe80::4e6d:58ff:fe87:1017, length 32
Out IP6 2a02:fb8::11 > fe80::6662:66ff:fe21:6b74: ICMP6, neighbor advertisement, tgt is fe80::4e6d:58ff:fe87:1017, length 24
In IP6 fe80::6662:66ff:fe21:6b74 > fe80::4e6d:58ff:fe87:1017: ICMP6, neighbor solicitation, who has fe80::4e6d:58ff:fe87:1017, length 32
Out IP6 2a02:fb8::11 > fe80::6662:66ff:fe21:6b74: ICMP6, neighbor advertisement, tgt is fe80::4e6d:58ff:fe87:1017, length 24
Out IP6 fe80::4e6d:58ff:fe87:1017 > ff02::1: ICMP6, router advertisement, length 32
In IP6 fe80::6662:66ff:fe21:6b74 > fe80::4e6d:58ff:fe87:1017: ICMP6, neighbor solicitation, who has fe80::4e6d:58ff:fe87:1017, length 32
Out IP6 2a02:fb8::11 > fe80::6662:66ff:fe21:6b74: ICMP6, neighbor advertisement, tgt is fe80::4e6d:58ff:fe87:1017, length 24
Out IP6 2a02:fb8::11 > fe80::6662:66ff:fe21:6b74: ICMP6, neighbor solicitation, who has fe80::6662:66ff:fe21:6b74, length 32
In IP6 2a06:61c1:ba35:10:6662:66ff:fe21:6b75 > 2a02:fb8::11: ICMP6, neighbor advertisement, tgt is fe80::6662:66ff:fe21:6b74, length 24
In IP6 fe80::6662:66ff:fe21:6b74 > fe80::4e6d:58ff:fe87:1017: ICMP6, neighbor solicitation, who has fe80::4e6d:58ff:fe87:1017, length 32
Out IP6 2a02:fb8::11 > fe80::6662:66ff:fe21:6b74: ICMP6, neighbor advertisement, tgt is fe80::4e6d:58ff:fe87:1017, length 24
In IP6 fe80::6662:66ff:fe21:6b74 > fe80::4e6d:58ff:fe87:1017: ICMP6, neighbor solicitation, who has fe80::4e6d:58ff:fe87:1017, length 32
Out IP6 2a02:fb8::11 > fe80::6662:66ff:fe21:6b74: ICMP6, neighbor advertisement, tgt is fe80::4e6d:58ff:fe87:1017, length 24
In IP6 fe80::6662:66ff:fe21:6b74 > fe80::4e6d:58ff:fe87:1017: ICMP6, neighbor solicitation, who has fe80::4e6d:58ff:fe87:1017, length 32
Out IP6 2a02:fb8::11 > fe80::6662:66ff:fe21:6b74: ICMP6, neighbor advertisement, tgt is fe80::4e6d:58ff:fe87:1017, length 24
Out IP6 fe80::4e6d:58ff:fe87:1017 > ff02::1: ICMP6, router advertisement, length 32
In IP6 fe80::6662:66ff:fe21:6b74 > fe80::4e6d:58ff:fe87:1017: ICMP6, neighbor solicitation, who has fe80::4e6d:58ff:fe87:1017, length 32
Out IP6 2a02:fb8::11 > fe80::6662:66ff:fe21:6b74: ICMP6, neighbor advertisement, tgt is fe80::4e6d:58ff:fe87:1017, length 24
It's best to troubleshoot this further before opening a bug report.
I tried. Nothing. https://forum.netgate.com/topic/196962/gigaclear-ip6-lose-of-connectivity-after-exactly-5-minutes
IF I put a static route in to the first v6 hop, my v6 connection stays up, but it failed after ~36 hours so this is not a solution.
Please - you have many reports of pfSense users on Gigaclear all experiencing the same issue. None of us are impacted by this when we use alternative firewalls to test. Gigaclear network team says their liveness detection is triggered as they do not receive valid solicitations from Sense. What do you need from Gigaclear users to help resolve this?
Updated by John S 12 days ago
I'm also seeing this same issue, I am using 25.03 BETA. Unfortunately I can currently only recreate this issue when pfSense is a client and a Juniper switch as the router. I will continue to try and recreate this using only pfSense at both ends, howevever my observations are:
In this setup on the pfSense side, the WAN interface must only have a link-local address, no GUA, as only a /48 prefix deligation is offered up. All LAN interfaces must have an IPv6 address that is not from /48 prefix that was deligated from the WAN. When setup like this, any neighbor advertisements from pfsense will have a source IP from one of the LAN interfaces, not the link-local from the WAN. It's a convoluted setup, but this does appear to recreate the issue.
At first, neighborhood solication and advertisements flow normally. The Juniper switch responds using its GUA as the source ip for it's neighborhood advertisment packages, while pfSense (ends :705) transmits it's neighborhood adversitements using it's link-local address as source. Within the Network advertisment from the Juniper switch the ICMPv6 Option 1 correctly contains it's link-layer/MAC address and pfSense appears to translate this into a link-local address of the Juniper switch to refresh it's NDP table as that is the entry that is kept alive for 5 minutes . pfSense does not refresh any NDP entry for the GUA source address of the neighborhood advertisement received, it only updates what I presume is a remembered/calculated link-local address as I don't see where it's getting this from.
33 03:44:47.13 fe80::1c1e:54ff:fe8a:705 ff02::1:ff00:32 ICMPv6 86 Neighbor Solicitation for 2a02:fb8::32 from 1e:1e:54:8a:07:05
34 03:44:48.12 fe80::1c1e:54ff:fe8a:705 ff02::1:ff00:32 ICMPv6 86 Neighbor Solicitation for 2a02:fb8::32 from 1e:1e:54:8a:07:05
35 03:44:49.12 fe80::1c1e:54ff:fe8a:705 ff02::1:ff00:32 ICMPv6 86 Neighbor Solicitation for 2a02:fb8::32 from 1e:1e:54:8a:07:05
36 03:45:03.44 fe80::1c1e:54ff:fe8a:705 fe80::4a5a:dff:fe5a:f2b7 ICMPv6 86 Neighbor Solicitation for fe80::4a5a:dff:fe5a:f2b7 from 1e:1e:54:8a:07:05
37 03:45:03.45 2a02:fb8::32 fe80::1c1e:54ff:fe8a:705 ICMPv6 78 Neighbor Advertisement fe80::4a5a:dff:fe5a:f2b7 (rtr, sol)
I beleive this is constistant with the PCAP's supplied by the Original Poster.
After approx 5 minutes from when the DHCPv6 negotiates a /48 IPv6 prefix (No IPv6 address is requested nor provided for the pfSense WAN interface), the NDP entry for fe80::4a5a:dff:fe5a:f2b7 (the Juniper switch) goes stale and outgoing traffic is dropped. Both ends do continue to attempt to call out to each other repeatedly albeit using a multicast address now, this continues indefinitely and neither end replies with a neighborhood advertisement, as bellow:
120 16:44:22.86 2a02:fb8::32 ff02::1:ff8a:705 ICMPv6 86 Neighbor Solicitation for fe80::1c1e:54ff:fe8a:705 from 4a:5a:0d:5a:f2:b7
121 16:44:22.86 fe80::1c1e:54ff:fe8a:705 ff02::1:ff5a:f2b7 ICMPv6 86 Neighbor Solicitation for fe80::4a5a:dff:fe5a:f2b7 from 1e:1e:54:8a:07:05
Note the Juniper switch is using a GUA for the source address, no advertisement is sent via this interface in response to this, this is wrong as a advertisement back should be sent on this interface. I have a second IPv6 enabled WAN interface that is the default route.
The connection has now failed at this point. Saving and applying the pfSense WAN interface settings triggers the DHCPv6 prefix deligation yet again and everything repeats as above with the connection again failing after 5 minutes.
Another observation, during the initial DHCPv6 negotiation the following warning is written to pfSense system logs against the dhcpc6 application:
unknown or unexpected DHCP6 option opt_20, len 0Also observed, no DHCPv6 renewal is ever attempted by pfSense.
Replacing pfsense for a minimal install OpenWRT and with the nearest equivalent DHCPv6 configuration (Request only an IPv6 prefix, do not request an IPv6 address, DHCPv6 Prefix Delegation size /48, Send IPv6 prefix hint, Do not wait for a RA) and everthing is stable with connectivity/traffic flowing for over 24 hours without interruption.
I am aware this isn't a perfect set of instructions to recreate this issue, I continue to try and recreate using only pfSense at both ends. However, it is easily recreatable here.
Updated by quiet lion 10 days ago
There is a workaround for this; adding a VIP to the WAN interface allows network solicitations to be sent back - or rather, be received successfully.