DHCP6 client does not take any action if the interface IPv6 address changes during renewal
I recently started using T-Mobile 5G Home Internet. The gateway device you're required to use is almost completely unconfigurable. You can't even change DHCP or DNS settings, forget about having a bridge mode or any sort of DMZ or IP bypass. So for IPv6 I need to use NAT, which actually works pretty well.
The problem is that occasionally the IPv6 address will change without the interface going down. First the Router Advertisements change and NAT and dpinger break. Eventually dhcp6c notices the change but does nothing about it:
Mar 11 03:44:26 router dhcp6c: Sending Renew Mar 11 03:44:27 router dhcp6c: dhcp6c Received INFO Mar 11 03:44:27 router dhcp6c: remove an address 2607:fb90:5128:15d0:24dc:4e2:d0b6:df11/128 on igb2 Mar 11 03:44:27 router dhcp6c: add an address 2607:fb90:5120:e987:24dc:4e2:d0b6:df11/128 on igb2 Mar 11 03:44:27 router dhcp6c: T1(1125) and/or T2(1800) is locally determined
Manually restarting dpinger and reloading the filters gets things working again, but what probably really needs to happen is to run /etc/rc.newwanipv6.
I'm thinking of patching (the generation of) /var/etc/dhcp6c_opt1_script.sh to run /etc/rc.newwanipv6 if the current interface IPv6 address does not match /var/db/opt1_ipv6 after a DHCPv6 INFO or RENEW. Is this a reasonable short-term fix?
Updated by Jim Pingle 3 months ago
- Assignee set to Jim Pingle
- Target version set to 2.7.0
- Plus Target Version set to 22.05
For that to trigger the client would have to fire the script during an event when the change occurs. It may not, but it's hard to say for sure based on the logs you have. For starters, go to System > Advanced on the Networking tab and check "DHCP6 Debug" and see what it logs at the time.
It's possible that would get triggered by the script at the RENEW case but it's not certain. The next problem is that at least according to the documentation, it doesn't look like the script will get any environment variables populated with data it could leverage to compare old/new IP addresses similar to what is done for the IPv4 client. It might be viable to read the address out of the
/var/db/<if>_ipv6 file and compare, like you suggest, but it would require testing to confirm.
Updated by Jim Pingle 3 months ago
- File dhcp6c-renew-12947.diff dhcp6c-renew-12947.diff added
- Assignee deleted (
- Target version deleted (
- Plus Target Version deleted (
I tried altering the script so it would fire during a renew with mixed success. Though I found another odd behavior. If I change the IP address assigned to the device in upstream DHCP, the next renew it picks up the new address. However, it gets added to the interface, so both the old and new are present. The old address isn't removed until its lease expires, even though it's supposed to be using the new address instead. It's marked deprecated in the meantime:
vtnet0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 description: WAN options=800b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,LINKSTATE> ether ca:1d:62:6c:c6:9c inet6 fe80::c81d:62ff:fe6c:c69c%vtnet0 prefixlen 64 scopeid 0x1 inet6 2001:db8::103:1 prefixlen 128 deprecated inet6 2001:db8::103:2 prefixlen 128 inet 198.51.100.103 netmask 0xffffff00 broadcast 198.51.100.255 media: Ethernet 10Gbase-T <full-duplex> status: active nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
The functions that fetch and check the interface address only get the old address which remains on the interface, not the additional address that came through.
When picking up the new address dhcp6c also seems to get stuck in a bit of a loop of REQUEST and RENEW messages (which I've seen be anywhere from ~15 seconds apart to ~60 seconds apart), but it may be related to the changes made here and not something else. It's been an issue in the past, though, see #11100 and #9634
Attached is a patch to try, though it's not very well optimized, it should at least show a difference in behavior.
Clearing the targets and assignment since there is a bit more happening here than I thought.
Updated by David Myers 3 months ago
The patch didn't work.
I applied the patch to my 2.5.2 system then enabled DHCP6 client debug mode and saved the interface in order to force the dhcp6c client script to get regenerated. I did not reboot.
The IPv6 address changed at approximately 13:07 in the attached logs, which are edited from about 13:00 until rc.newwanipv6 ran a couple times, and with DHCPv4 messages removed.
One thing I don't understand is why there are no messages in the logs from the calls to logger from the dhcp6c client script. The logger command works fine from the command line.
While IPv6 was not working ifconfig showed this:
igb2: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1420 description: LTE options=e100bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWFILTER,RXCSUM_IPV6,TXCSUM_IPV6> ether 00:e0:67:22:fc:52 inet6 fe80::2e0:67ff:fe22:fc52%igb2 prefixlen 64 scopeid 0x3 inet6 2607:fb90:5120:e987:2e0:67ff:fe22:fc52 prefixlen 64 detached deprecated autoconf inet6 2607:fb90:1b7d:9992:2e0:67ff:fe22:fc52 prefixlen 64 autoconf inet6 2607:fb90:1b7d:9992:24dc:4e2:d0b6:df11 prefixlen 128 inet 192.168.12.245 netmask 0xffffff00 broadcast 192.168.12.255 media: Ethernet autoselect (1000baseT <full-duplex>) status: active nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
In order to recover IPv6 I needed to release and renew the interface, at which point it looked like this:
igb2: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1420 description: LTE options=e100bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWFILTER,RXCSUM_IPV6,TXCSUM_IPV6> ether 00:e0:67:22:fc:52 inet6 fe80::2e0:67ff:fe22:fc52%igb2 prefixlen 64 scopeid 0x3 inet6 2607:fb90:1b7d:9992:2e0:67ff:fe22:fc52 prefixlen 64 autoconf inet6 2607:fb90:1b7d:9992:24dc:4e2:d0b6:df11 prefixlen 128 inet 192.168.12.245 netmask 0xffffff00 broadcast 192.168.12.255 media: Ethernet autoselect (1000baseT <full-duplex>) status: active nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
I hadn't looked at ifconfig before and was surprised to see two addresses. pfSense uses the first one.
I've backed out the patch.
Updated by → luckman212 10 days ago
It appears we are out of luck on having
devd fire events for IP address changes. There is a commit: https://reviews.freebsd.org/rGa75819461ec7c7d8468498362f9104637ff7c9e9 but that seems like it might not even be in FreeBSD13 (much less 12.3...). Maybe we can get this backported to 12.x somehow? Seems like it would be immensely useful in these cases.
My poor man's solution is hashing the output of
ifconfig on all DHCP6 interfaces with cron and firing an event if the hash changes. But this places unnecessary load on the system and also has a potential for up to 1 minute of downtime before the change is picked up. Still, it's the best I can come up with for now until devd can trigger on address changes. I have a PR for this that I will submit later today.
Updated by → luckman212 2 days ago
I posted on the PR that since Reid Linnemann has just deprecated pfSense_getall_interface_addresses(), this should probably be updated to use the new pfSense_get_ifaddrs() function, where possible.
However, I do want to note for the record that my patch has been running for over a week now and has 100% fixed my issue.