Bug #8883
closedDefault route order set in Routing not honored
0%
Description
I have two interfaces:
- xn0: 10.0.0.253, gateway 10.0.0.1 (WAN)
- xn1: 10.0.0.254, gateway 10.0.0.1 (WAN2)
If I set the default gateway to xn1 (WAN2) in the Gateways GUI menu, netstat -rn still shows the default route set to xn0.
I'm using shellcmd to force the change on boot, but it reverts every few hours.
Updated by Jaime Geiger over 6 years ago
I propose the following fix on /etc/int/system.inc line 755:
if (!empty($interface) && $interface != $interfacegw) { ; } else if (is_ipaddrv4($gatewayip)) { $realgwif = get_real_interface($interfacegw); log_error(sprintf(gettext("ROUTING: setting default route to %s via %s"), $gatewayip, $realgwif)); route_add_or_change("-inet default {$gatewayip} -ifp {$realgwif}"); }
For reference, pre change:
if (!empty($interface) && $interface != $interfacegw) { ; } else if (is_ipaddrv4($gatewayip)) { log_error(sprintf(gettext("ROUTING: setting default route to %s"), $gatewayip)); route_add_or_change("-inet default {$gatewayip}"); }
The default behavior of the route add command if an IP is specified is to pick the lowest numbered interface. If both interfaces have the same IP addresses it will always pick xn0, causing this bug. This fix just specifies forces the interface to be set.
Might want to do this a little further down for ipv6 as well...
if (!empty($interface) && $interface != $interfacegwv6) { ; } else if (is_ipaddrv6($gatewayipv6)) { $ifscope = ""; if (is_linklocal($gatewayipv6) && !strpos($gatewayipv6, '%')) { $ifscope = "%{$defaultifv6}"; } log_error(sprintf(gettext("ROUTING: setting IPv6 default route to %s"), $gatewayipv6 . $ifscope)); route_add_or_change("-inet6 default {$gatewayipv6}{$ifscope}"); }
Updated by Jim Pingle over 6 years ago
- Status changed from New to Not a Bug
Having two interfaces in the same subnet is not supported, nor is having two interfaces with the same gateway.
The behavior of the underlying operating system will be a problem here beyond anything in our code.
Updated by Jaime Geiger over 6 years ago
I have created a fix (above) so clearly there can be something done in your code. The fix does not break current configurations.
I am in AWS, so that's the reason for this weird use case. I wouldn't do it like this normally, trust me.
Updated by Jim Pingle over 6 years ago
It may appear to "work" for you but it is not a general fix that will work across all platforms, and may not behave consistently for you over time. Having multiple interfaces in the same network is a broken design that should not be relied upon. The gateway cannot work on both interfaces as the ARP entry will only be on one of them, and that's where all of the traffic will go. The OS will stop using one or the other interface depending on what happens when the ARP entries expire and which interface picks them up the next time. Traffic may behave asymmetrically since the upstream may deliver some things inbound on one interface but replies will exit the other.
If it happens to function at all, it's out of sheer luck and coincidence, not because of any changes in the code.
Updated by Jaime Geiger over 6 years ago
Thanks for your time. I'll just keep the changes locally.
Updated by Jaime Geiger over 6 years ago
Though, as food for thought:
arp -a ... ip-10-0-0-1.ec2.internal (10.0.0.1) at 0a:f4:bc:bb:aa:cc on xn1 expires in 1188 seconds [ethernet] ip-10-0-0-1.ec2.internal (10.0.0.1) at 0a:f4:bc:bb:aa:cc on xn0 expires in 1189 seconds [ethernet] ...
and
netstat -rn Routing tables Internet: Destination Gateway Flags Netif Expire default 10.0.0.1 UGS xn1 ...
The arp entry for the gateway is on both, and the routing table relies on the interface here (via the -ifp option).
I think this is just how the kernel works, but you have worked with more platforms on this so you may be right on the portability part, since I don't have much frame of reference outside amd64.