Project

General

Profile

Actions

Bug #8883

closed

Default route order set in Routing not honored

Added by Jaime Geiger over 6 years ago. Updated over 6 years ago.

Status:
Not a Bug
Priority:
Normal
Assignee:
-
Category:
Routing
Target version:
-
Start date:
09/11/2018
Due date:
% Done:

0%

Estimated time:
Plus Target Version:
Release Notes:
Affected Version:
2.4.2_1
Affected Architecture:

Description

I have two interfaces:
- xn0: 10.0.0.253, gateway 10.0.0.1 (WAN)
- xn1: 10.0.0.254, gateway 10.0.0.1 (WAN2)

If I set the default gateway to xn1 (WAN2) in the Gateways GUI menu, netstat -rn still shows the default route set to xn0.
I'm using shellcmd to force the change on boot, but it reverts every few hours.

Actions #1

Updated by Jaime Geiger over 6 years ago

this is on amd64, by the way.

Actions #2

Updated by Jaime Geiger over 6 years ago

I propose the following fix on /etc/int/system.inc line 755:

                if (!empty($interface) && $interface != $interfacegw) {
                        ;
                } else if (is_ipaddrv4($gatewayip)) {
                        $realgwif = get_real_interface($interfacegw);
                        log_error(sprintf(gettext("ROUTING: setting default route to %s via %s"), $gatewayip, $realgwif));
                        route_add_or_change("-inet default {$gatewayip} -ifp {$realgwif}");
                }

For reference, pre change:

                if (!empty($interface) && $interface != $interfacegw) {
                        ;
                } else if (is_ipaddrv4($gatewayip)) {
                        log_error(sprintf(gettext("ROUTING: setting default route to %s"), $gatewayip));
                        route_add_or_change("-inet default {$gatewayip}");
                }

The default behavior of the route add command if an IP is specified is to pick the lowest numbered interface. If both interfaces have the same IP addresses it will always pick xn0, causing this bug. This fix just specifies forces the interface to be set.

Might want to do this a little further down for ipv6 as well...

                if (!empty($interface) && $interface != $interfacegwv6) {
                        ;
                } else if (is_ipaddrv6($gatewayipv6)) {
                        $ifscope = "";
                        if (is_linklocal($gatewayipv6) && !strpos($gatewayipv6, '%')) {
                                $ifscope = "%{$defaultifv6}";
                        }
                        log_error(sprintf(gettext("ROUTING: setting IPv6 default route to %s"), $gatewayipv6 . $ifscope));
                        route_add_or_change("-inet6 default {$gatewayipv6}{$ifscope}");
                }

Actions #3

Updated by Jim Pingle over 6 years ago

  • Status changed from New to Not a Bug

Having two interfaces in the same subnet is not supported, nor is having two interfaces with the same gateway.

The behavior of the underlying operating system will be a problem here beyond anything in our code.

Actions #4

Updated by Jaime Geiger over 6 years ago

I have created a fix (above) so clearly there can be something done in your code. The fix does not break current configurations.
I am in AWS, so that's the reason for this weird use case. I wouldn't do it like this normally, trust me.

Actions #5

Updated by Jim Pingle over 6 years ago

It may appear to "work" for you but it is not a general fix that will work across all platforms, and may not behave consistently for you over time. Having multiple interfaces in the same network is a broken design that should not be relied upon. The gateway cannot work on both interfaces as the ARP entry will only be on one of them, and that's where all of the traffic will go. The OS will stop using one or the other interface depending on what happens when the ARP entries expire and which interface picks them up the next time. Traffic may behave asymmetrically since the upstream may deliver some things inbound on one interface but replies will exit the other.

If it happens to function at all, it's out of sheer luck and coincidence, not because of any changes in the code.

Actions #6

Updated by Jaime Geiger over 6 years ago

Thanks for your time. I'll just keep the changes locally.

Actions #7

Updated by Jaime Geiger over 6 years ago

Though, as food for thought:

arp -a
...
ip-10-0-0-1.ec2.internal (10.0.0.1) at 0a:f4:bc:bb:aa:cc on xn1 expires in 1188 seconds [ethernet]
ip-10-0-0-1.ec2.internal (10.0.0.1) at 0a:f4:bc:bb:aa:cc on xn0 expires in 1189 seconds [ethernet]
...

and
netstat -rn
Routing tables

Internet:
Destination        Gateway            Flags     Netif Expire
default            10.0.0.1           UGS         xn1
...

The arp entry for the gateway is on both, and the routing table relies on the interface here (via the -ifp option).
I think this is just how the kernel works, but you have worked with more platforms on this so you may be right on the portability part, since I don't have much frame of reference outside amd64.

Actions

Also available in: Atom PDF