pfSense

Summary¶

Reconfiguring an assigned interface re-applies its IP address even when nothing about it has
changed. The address is briefly removed and re-added (~3 ms). While it is gone, the kernel purges
every route whose nexthop resolved through that subnet -- and emits no routing-socket message for
those purged routes. Any routing daemon holding them (FRR here) is never told, keeps them marked
FIB-installed, and never reinstalls them.

The routes are gone from the kernel and never come back. This silently severs connectivity to
every network reachable via that interface, with no alarm and no self-recovery, while every status
view still reports healthy -- FRR shows "Established" with correct prefix counts throughout. Only
"netstat -rn" reveals it. Verified still broken after 8 minutes; manual intervention is always
required.

Reproducer¶

# 1. Run FRR/BGP over an assigned interface; confirm its routes are in the kernel:
netstat -rn -f inet | grep <iface>

# 2. Reconfigure that interface -- either is sufficient:
#      Interfaces > OPTx > Save > Apply
#      php: interface_configure("optX", true);

# 3. Routes are now gone from the kernel, but FRR still believes they are installed:
netstat -rn -f inet | grep <iface>           # missing
vtysh -c "show ip route bgp"                 # still shows B>*
vtysh -c "show bgp ipv4 unicast summary"     # Established, correct PfxRcd

An interface carrying 8 BGP routes drops to 0 and stays there. "Apply to all" (what creating a new
tunnel does) strands every BGP route on the box at once.

Root cause¶

Traced at runtime by instrumenting /etc/inc/interfaces.inc with log_error():

interfaces.inc:4392   pfSense_interface_setaddress($realif, "{$wancfg['ipaddr']}/{$wancfg['subnet']}");

This re-applies the address unconditionally, with no check for whether it is already configured
with exactly that value. Re-applying an identical address still produces ADDR_DEL followed by
ADDR_ADD at the kernel level, and that momentary removal is what purges the routes.

A bare interface_configure() hits ONLY :4392 and is sufficient to cause the fault --
interface_bring_down() is deliberately skipped for configured interfaces (:4199). A package Apply
additionally hits interface_bring_down():1937 ("ifconfig {$realif} delete") before :4392.

Confirmed with "route -n monitor", which taps the routing socket independently of FRR:

18:05:18.136  delete   addr   192.168.135.4/31  iface tun_wg2
18:05:18.139  add/repl addr   192.168.135.4/31  iface tun_wg2     <-- 3 ms later

The route that was actually lost (192.168.80.0/22 via 192.168.135.5) appears NOWHERE in that
capture, though the kernel did announce three unrelated prefixes in the same 200 ms window -- so
the socket is working, the purged route is simply never announced.

Nothing notices because the interface never goes down (zero RTM_IFANNOUNCE for the whole
operation), so FRR's fast-external-failover -- which keys on interface state -- does not fire, and
a 3 ms blink is far below BGP's hold timer. Destroying the interface is actually safe, because it
is loud enough to trigger BGP's own recovery; the gentle no-op re-apply is the damaging one.

Suggested fix¶

Skip the re-application when the address already matches: compare against what is currently
configured before calling pfSense_interface_setaddress() at :4392, or make that call a no-op when
the value is unchanged. Since the address is being rewritten with an identical value, skipping it
is a no-op in every other respect. The same guard is worth applying at :1937.

Notes¶

Tested with FRR/BGP over WireGuard on three systems (1, 2 and 3 tunnels), via both a package Apply
and a bare interface_configure() with no package involved. The mechanism should affect any assigned
interface carrying daemon-installed routes (IPsec VTI, OSPF over VLANs), though I have only
measured the BGP case. pfSense re-adds its own connected and static routes, so only daemon-installed
routes are orphaned -- likely why this has gone unnoticed.

Workaround: "clear bgp <peer>" (hard) forces the withdrawal that never happened. A soft clear does
nothing. We automated it with a devd hook on IFNET/ADDR_ADD, which heals in ~3 s.

Bug #14668 (pfSense Packages > FRR) is this same defect seen through WireGuard -- open since
2023-08 with no root cause. Filing here because the trigger is not WireGuard-specific and the fix
is in core.

Environment¶

pfSense 2.8.1-RELEASE (FreeBSD 15.0-CURRENT)
pfSense-pkg-frr 2.0.2_6 / frr9-9.1.2_1 (FRRouting 9.1.2)
pfSense-pkg-WireGuard 0.2.9_6