Project

General

Profile

Actions

Bug #8815

open

IP addresses are removed from interfaces when link is lost and either IPv4 or IPv6 is dynamic

Added by Jim Pingle about 6 years ago. Updated about 3 years ago.

Status:
New
Priority:
Normal
Assignee:
Category:
Interfaces
Target version:
Start date:
08/20/2018
Due date:
% Done:

0%

Estimated time:
Plus Target Version:
Plus-Next
Release Notes:
Affected Version:
2.4.x
Affected Architecture:
All

Description

When an interface loses link the IPv4 addresses, including VIPs, can disappear from the interface. If the link is down when the interface settings are applied, the address is present, but plugging in the interface and then removing the link again can make the addresses disappear.

The behavior is not consistent across all configurations and hardware. For example, it appears to happen predictably on igb(4) interfaces but not vmx(4) interfaces, and in some cases re(4) is OK and fails for others.

The main consequence is that anything bound to or relying on those addresses will fail when they disappear. The highest impact is from HA/CARP. The VIPs disappear from the interface, so the OS does not trigger demotion as expected with link loss.

IPv6 addresses, if any exist, are still present on the interfaces after link is lost.

For example, on an SG-5100. LAN is unplugged before the test:

igb1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=6400bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
    ether 00:90:0b:7a:8a:66
    hwaddr 00:90:0b:7a:8a:66
    inet 10.23.0.1 netmask 0xffffff00 broadcast 10.23.0.255 
    inet6 2001:db8:1:ee10:290:bff:fe7a:8a66 prefixlen 60 
    inet6 fe80::1:1%igb1 prefixlen 64 scopeid 0x2 
    nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
    media: Ethernet autoselect
    status: no carrier

Plug in LAN, then unplug LAN:

igb1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=6400bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
    ether 00:90:0b:7a:8a:66
    hwaddr 00:90:0b:7a:8a:66
    inet6 2001:db8:1:ee10:290:bff:fe7a:8a66 prefixlen 60 
    inet6 fe80::1:1%igb1 prefixlen 64 scopeid 0x2 
    nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
    media: Ethernet autoselect
    status: no carrier

Logs for igb1 during the transition:

Aug 20 11:54:29 missy kernel: igb1: link state changed to UP
Aug 20 11:54:29 missy check_reload_status: Linkup starting igb1
Aug 20 11:54:29 missy check_reload_status: Reloading filter
Aug 20 11:54:30 missy php-fpm[585]: /rc.linkup: DEVD Ethernet attached event for lan
Aug 20 11:54:30 missy php-fpm[585]: /rc.linkup: HOTPLUG: Configuring interface lan
Aug 20 11:54:30 missy check_reload_status: Restarting ipsec tunnels
Aug 20 11:54:33 missy check_reload_status: Linkup starting igb1
Aug 20 11:54:33 missy kernel: igb1: link state changed to DOWN
Aug 20 11:54:34 missy php-fpm[584]: /rc.newwanip: The command '/usr/local/sbin/unbound -c /var/unbound/unbound.conf' returned exit code '1', the output was '[1534780474] unbound[62993:0] error: bind: address already in use [1534780474] unbound[62993:0] fatal error: could not open ports' 
Aug 20 11:54:34 missy check_reload_status: updating dyndns lan
Aug 20 11:54:34 missy php-fpm[39495]: /rc.linkup: DEVD Ethernet detached event for lan
Aug 20 11:54:35 missy check_reload_status: Reloading filter

Actions #1

Updated by Steve Wheeler about 6 years ago

I have replicated this in em(4) and re(4) (on an APU) but not on mvneta(4).
All on the latest 2.4.4 snapshot.

Actions #2

Updated by Anonymous about 6 years ago

  • Assignee set to Luiz Souza
Actions #3

Updated by Anonymous about 6 years ago

  • Status changed from New to 13
Actions #4

Updated by Anonymous about 6 years ago

  • Status changed from 13 to In Progress
Actions #5

Updated by Anonymous about 6 years ago

  • Target version changed from 2.4.4 to 2.4.4-GS
Actions #6

Updated by Luiz Souza about 6 years ago

  • Status changed from In Progress to 13
Actions #7

Updated by Luiz Souza about 6 years ago

  • Status changed from 13 to New
Actions #8

Updated by Anonymous about 6 years ago

  • Target version changed from 2.4.4-GS to 2.4.4-p1
Actions #9

Updated by Anonymous almost 6 years ago

  • Target version changed from 2.4.4-p1 to 48
Actions #10

Updated by Jim Pingle over 5 years ago

  • Target version changed from 48 to 2.5.0
Actions #11

Updated by Anonymous almost 4 years ago

  • Target version changed from 2.5.0 to CE-Next
Actions #13

Updated by Steve Wheeler about 3 years ago

This appears to be coupled to the IPv6 setting on the interface that is disconnected.

Testing here against 2.5.2, with the LAN set to 'Track Interface' and tracking WAN which is DHCPv6 the IP addresses on the interface are lost when it is disconnected.

With it set to None or as a valid Static IPv6 address all IP addresses are retained.

Actions #14

Updated by Jim Pingle about 3 years ago

  • Subject changed from IPv4 addresses disappear from interfaces when link is lost to IP addresses are removed from interfaces when link is lost and either IPv4 or IPv6 is dynamic
  • Priority changed from Very High to Normal
  • Plus Target Version set to Plus-Next

This appears to be due to an interface having either IPv4 or IPv6 set to a dynamic type (read: Not static, not 'none'). If both are static or one is static and the other set to 'none' this doesn't happen.

In the case where either type of address is dynamic a link loss triggers a different code path which runs interface_bring_down() on the interface which lost link. That function in turn runs ifconfig <name> delete which clears the IPv4 addresses and other functions which have similar effects, the end result being that all addresses are cleared from the interface when it goes down.

For typical cases this is fairly low impact since the addresses come back when the interface regains link. However, if a system is part of an HA cluster using CARP VIPs, this can be a problem. When the CARP VIP is removed, rather than kept in place, it causes problems with demotion and preemption so failover won't function.

Dynamic addresses are not compatible with HA, so HA nodes should not be configured in this way. If IPv6 is not going to be used, then all interfaces should have IPv6 set to "None" rather than leaving DHCPv6 or Track Interface selected.

That said, it might be nice to detect such cases and not be so harsh. For example, maybe when link is lost, only addresses of the dynamic type (IPv4 or IPv6) are cleared when mixing static and dynamic on an interface.

Updating subject and lowering priority to reflect what we've found so far.

Actions

Also available in: Atom PDF