pfSense » pfSense Packages

I am reproducing this issue (constantly changing FRR running configuration leading to inactive routes) and unfortunately, it's still present in 2.6RC. I'm not sure if it's relevant, but we're trying to send/receive OSPF routes over an OpenVPN tunnel. Our setup worked great with 2.4.5_p1 and several previous versions, but firewalls upgraded to 2.5.x have had issues with routes occasionally going inactive in the Zebra table and thus not being in the system routing table presumably due to this issue and probably some WAN instability mixed in. I can get things back to a working state by restarting FRR Zebra (or interestingly enough if I go to Raw Config and hit "Update Running Config", that stabilizes things as well), but obviously that's manual and shouldn't be necessary.

I have a TAC case open about the issue right now as well because staying on old versions isn't something we want to do forever. They confirmed (as is seen in Jim Pingle's Dynamic Routing with FRR Hangout) that having ACCEPTFILTER checked on both my LAN interface and the OpenVPN interface is advisable. They also recommended adding my VPN network as a static localhost route under FRR Global/Zebra Route Handling which I did, but it wasn't helpful to avoid the bad behavior. They suggested trying 2.6 RC, but the behavior is the same.

I am reproducing it by inserting a switch in between the WAN interface of my pfSense box and my actual WAN connection (cable modem). To reproduce, I disable the interface on the switch that the cable modem is hooked to and reboot the firewall. Once its up, I took a peak at the show running-configuration in vtysh and it was stable. Upon re-enabling the cable modem connection, the routes I expect to populate end up in Zebra, but Inactive and they are NOT placed into the system routing table. At this point, looking at show running configuration repeatedly shows the ACCEPTFILTER section constantly changing and it never settles until I restart Zebra.

I'm happy to provide what I can to try to get it fixed. I thought sharing my method for reproducing it VERY consistently might be helpful and I'm happy to provide additional information that would be helpful. I'm not a developer or an OSPF/FRR expert, but I do have a test setup specifically for this right now and I'm happy to work with someone.

Regarding ACCEPTFILTER, you can test the patch here listed on #11686

It looks to me like, with the patch, the "seq xx" numbering has been corrected so that the "permit any" is always the highest number which should place it at the end, however, I'm also still seeing the weird constantly changing/cycling config in VTYSH when doing show running-configuration where it looks like the permit any keeps moving up through the other entries. I'm unsure if this has any impact on stability, but it is definitely weird and seems not right. I originally complained about this issue because I'd have a situation where all of my remote routes would end up as Inactive and therefore no way to get to them. That's what I am trying to avoid. Maybe leaving ACCEPTFILTER unchecked is the answer, but the Jim Pingle Hangout and TAC have guided me to check it for both LAN networks and VPN tunnel networks.

Matthew D wrote in #note-7:

It looks to me like, with the patch, the "seq xx" numbering has been corrected so that the "permit any" is always the highest number which should place it at the end, however, I'm also still seeing the weird constantly changing/cycling config in VTYSH when doing show running-configuration

Am seeing the same thing as you and thinking the same thing. The ACCEPTFILTER seems to be working correctly for me now, but the output of the prefix-list is still weird.
Here's a grab from my "show running-config":

ip prefix-list CONNECT seq 10 permit 10.27.194.0/24
ip prefix-list ACCEPTFILTER seq 50 deny 10.255.197.0/30
ip prefix-list ACCEPTFILTER seq 60 deny 10.255.197.1/32
ip prefix-list ACCEPTFILTER seq 70 permit any
ip prefix-list ACCEPTFILTER seq 10 deny 10.255.195.0/30
ip prefix-list ACCEPTFILTER seq 20 deny 10.255.195.1/32
ip prefix-list ACCEPTFILTER seq 30 deny 10.255.196.0/30
ip prefix-list ACCEPTFILTER seq 40 deny 10.255.196.1/32
!

Run it again I get:

ip prefix-list CONNECT seq 10 permit 10.27.194.0/24
ip prefix-list ACCEPTFILTER seq 60 deny 10.255.197.1/32
ip prefix-list ACCEPTFILTER seq 70 permit any
ip prefix-list ACCEPTFILTER seq 10 deny 10.255.195.0/30
ip prefix-list ACCEPTFILTER seq 20 deny 10.255.195.1/32
ip prefix-list ACCEPTFILTER seq 30 deny 10.255.196.0/30
ip prefix-list ACCEPTFILTER seq 40 deny 10.255.196.1/32
ip prefix-list ACCEPTFILTER seq 50 deny 10.255.197.0/30

It's not actually random. No matter how log I wait - the next time I run "show running-config", the "seq 10" moves up one, and the entry that was previously listed first (seq 50 in my example), wraps around to the bottom. It might be just cosmetic, but a proper display output should always show that "seq 10" entry first.

When I generate the FRR config via the GUI, it actually formats and displays the ACCEPTFILTER correctly, so the above just seems to be a weird CLI display issue.