pfSense

We've spent some time troubleshooting our inability to set an MTU value on a LAGG interface and have found what we believe to be a bug. The LAGG we're using is very basic, just a LACP bond of two NIC ports to access mode ports on a pair of switches. We tracked the problem to the /etc/inc/interfaces.inc file, the relevant sections of which I've included, with some comments below:

function interface_lagg_configure($lagg) {
...
    $lagg_mtu = interface_find_child_cfgmtu($laggif);
...
}

...

function interface_find_child_cfgmtu($realiface) {
...
    // Get the friendly name of the lagg, e.g. lagg0 becomes opt1
    $interface = convert_real_interface_to_friendly_interface_name($realiface);
    // Get VLANs attached to the adapter or empty if none
    $vlans = link_interface_to_vlans($realiface);
    // Get the Bridge config attached to the adapter or empty if none
    $bridge = link_interface_to_bridge($realiface);
...
    // Skipping GIF/GRE config for bevity
...
    $mtu = 0;
    // If one or more of the VLANs have an MTU, get the biggest one
    if (is_array($vlans)) {
        foreach ($vlans as $vlan) {
            $ifass = convert_real_interface_to_friendly_interface_name($vlan['vlanif']);
            if (empty($ifass))
                continue;
            if (!empty($config['interfaces'][$ifass]['mtu'])) {
                if (intval($config['interfaces'][$ifass]['mtu']) > $mtu)
                    $mtu = intval($config['interfaces'][$ifass]['mtu']);
            }
        }
    }
...
    // Repeat for GIF/GRE, resetting the MTU to the largest found value
...
    // Get the bridge (if there is one) and find the MTU for it, replacing the current MTU value if this one is larger
    $ifass = convert_real_interface_to_friendly_interface_name($bridge);
    if (!empty($ifass) && !empty($config['interfaces'][$ifass]['mtu'])) {
        if (intval($config['interfaces'][$ifass]['mtu']) > $mtu)
            $mtu = intval($config['interfaces'][$ifass]['mtu']);
    }
    unset($vlans, $bridge, $gifs, $gres, $ifass, $vlan);

    // Return the largest MTU value of any attached VLAN, Bridge, GIF, or GRE configs attached to this interface
    return $mtu;
}

The way we read this is that the MTU value for your LAGG interface will only be set if you have a sub-interface (VLAN, Bridge, GIF, or GRE) attached to it. In our case, we just want the MTU value we set on the interface backed by the LAGG to be used. We resolved this by modifying the 'interface_find_child_cfgmtu' method include the actual interface in the search for an MTU. A sample is below:

--- interfaces.inc.a    2015-03-11 08:07:32.061604415 -0400
+++ interfaces.inc.b    2015-03-12 06:46:05.822575326 -0400
@@ -4548,11 +4548,18 @@
             }
         }
     }
-    $ifass = convert_real_interface_to_friendly_interface_name($bridge);
-    if (!empty($ifass) && !empty($config['interfaces'][$ifass]['mtu'])) {
-        if (intval($config['interfaces'][$ifass]['mtu']) > $mtu)
-            $mtu = intval($config['interfaces'][$ifass]['mtu']);
+    if (is_empty($bridge)) {
+        $ifass = convert_real_interface_to_friendly_interface_name($bridge);
+        if (!empty($ifass) && !empty($config['interfaces'][$ifass]['mtu'])) {
+            if (intval($config['interfaces'][$ifass]['mtu']) > $mtu)
+                $mtu = intval($config['interfaces'][$ifass]['mtu']);
+        }
+    }
+    if (!empty($config['interfaces'][$interface]['mtu'])) {
+        if (intval($config['interfaces'][$interface]['mtu']) > $mtu)
+            $mtu = intval($config['interfaces'][$interface]['mtu']);
     }
+
     unset($vlans, $bridge, $gifs, $gres, $ifass, $vlan);

     return $mtu;

We're not submitting a pull request for this because we're not entirely sure if there are ramifications to making this change that are not obvious. We understand trying to find the largest MTU used by a VLAN trunked to an interface as that is likely to be the actual interface MTU, but it seems like this should only be a fall back if the interface MTU is not specified. The only other reference to this 'interface_find_child_cfgmtu' that we see in the interface code is for bridge interface setup. If it seems better to keep the MTU checking code specific to the LAGG interfaces, a modification of 'interface_lagg_configure' might be better:

--- interfaces.inc.a    2015-03-11 08:07:32.061604415 -0400
+++ interfaces.inc.c    2015-03-12 06:59:20.090545811 -0400
@@ -671,7 +671,17 @@
         $laggif = pfSense_interface_create("lagg");

     /* Check if MTU was defined for this lagg interface */
-    $lagg_mtu = interface_find_child_cfgmtu($laggif);
+    $lagg_mtu = 0;
+    $friendly_iface = convert_real_interface_to_friendly_interface_name($laggif);
+    if (!empty($friendly_iface) && !empty($config['interfaces'][$friendly_iface]['mtu'])) {
+        /* Use the actual interface MTU if we have it */
+        if (intval($config['interfaces'][$friendly_iface]['mtu']) > $lagg_mtu)
+            $lagg_mtu = intval($config['interfaces'][$friendly_iface]['mtu']);
+    }
+    if ($lagg_mtu == 0) {
+        /* If the actual interface doesn't have an MTU, check any child interfaces and find the biggest MTU */
+        $lagg_mtu = interface_find_child_cfgmtu($laggif);
+    }
     if ($lagg_mtu == 0) {
         /* Calculate smaller mtu and enforce it */
         $smallermtu = 0;

My PHP is rather rusty, so please pardon the code. I will say that we're using something like the first patch successfully now. We did not test whether MTU changes worked on a running system. The change does correctly set MTU value on both the LAGG interface and the underlying adapters when the system boots.