pfSense

Syslogd is started before any packages are started, including the FRR package. If any remote syslog servers are on a network whose route is learned over BGP, then this traffic will be routed to the default gateway initially. This is expected behavior, since the FRR package hasn't been loaded and no BGP routes have been received.

The problem is that the traffic is NOT being redirected after BGP routes are established due to the state that was created initially by routing the traffic through the default GW.

In my specific case, I've got a remote site sending syslog traffic over an OpenVPN tunnel with BGP routing between sites. When the remote router reboots, the syslog messages are routed out of the WAN interface, creating a state with an "src-org" of the LAN IP and "src" of the WAN IP. After the FRR package starts and the BGP routes are received, these messages continue to go out of the WAN interface until the state is killed.

I originally reported this bug with #14403, but was told:

This is a configuration issue -- if traffic is taking a path you don't want when the VPN is down, you need to add rules to block it (e.g. reject it outbound on WAN via floating rules).

However, this does not work either. While it does prevent the traffic from exiting the WAN interface, the syslog messages are still not being routed properly after the BGP routes are received. This began occurring for me originally on 23.01, but is still occurring in 23.09.1.

I was able to get this working by adding some code in to the "/etc/rc.state_packages" script in the foreach loop that starts that packages that checks to see if the FRR package was just started, then looks to see if any remote syslog servers were configured. If there were any servers configured, then it sleeps for 15 seconds (to give time for the BGP peering to start) before looping through the servers and checking for any existing states. If any states exists, it checks for a "src-org" field and compares it to the "src" field. If the "src" and "src-org" don't match, then it kills that state. I have tested this change with 23.09.1, and it has been working as expected.