pfSense

What do "netstat -rn" and "ifconfig -a" look like before and after? Any notable differences?

Jim Pingle wrote:

What do "netstat -rn" and "ifconfig -a" look like before and after? Any notable differences?

Well yes:

1/ The default route is gone.

# diff -Nau /tmp/netstat_working.txt /tmp/netstat_broken.txt
--- /tmp/netstat_working.txt    2016-09-04 13:28:56.537548000 +0200
+++ /tmp/netstat_broken.txt     2016-09-04 13:31:19.587979000 +0200
@@ -2,7 +2,6 @@

 Internet:
 Destination        Gateway            Flags      Netif Expire
-default            10.20.31.1         UGS     bridge0
 10.20.31.0/24      link#10            U       bridge0
 10.20.31.254       link#10            UHS         lo0
 10.20.33.0/24      link#12            U       bridge1
@@ -11,7 +10,6 @@

 Internet6:
 Destination                       Gateway                       Flags      Netif Expire
-default                           2001:470:dead:beef::1           UGS     bridge0
 ::1                               link#8                        UH          lo0
 2001:470:dead:beef::/64             link#10                       U       bridge0
 2001:470:dead:beef::254             link#10                       UHS         lo0

2/ The bridge MAC address changes and the path cost gets bumped to ridiculous values:

# diff -Nau /tmp/ifconfig_working.txt /tmp/ifconfig_broken.txt
--- /tmp/ifconfig_working.txt   2016-09-04 13:29:11.212071000 +0200
+++ /tmp/ifconfig_broken.txt    2016-09-04 13:31:29.169965000 +0200
@@ -36,18 +36,6 @@
-bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
-       ether 02:b2:90:f6:ea:00
-       inet 10.20.31.254 netmask 0xffffff00 broadcast 10.20.31.255
-       inet6 2001:470:dead:beef::254 prefixlen 64
-       nd6 options=1<PERFORMNUD>
-       id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
-       maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
-       root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
-       member: vr0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
-               ifmaxaddr 0 port 1 priority 128 path cost 55
-       member: vr1 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
-               ifmaxaddr 0 port 2 priority 128 path cost 55
 ath0_wlan0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        ether 66:02:b4:7e:66:ea
        inet6 fe80::6402:b4ff:fe7e:66ea%ath0_wlan0 prefixlen 64 scopeid 0x9
@@ -71,3 +59,15 @@
+bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
+       ether 02:a0:aa:f2:78:00
+       inet 10.20.31.254 netmask 0xffffff00 broadcast 10.20.31.255
+       inet6 2001:470:dead:beef::254 prefixlen 64
+       nd6 options=1<PERFORMNUD>
+       id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
+       maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
+       root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
+       member: vr0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
+               ifmaxaddr 0 port 1 priority 128 path cost 200000
+       member: vr1 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
+               ifmaxaddr 0 port 2 priority 128 path cost 200000

I am having a similar issue whenever my WAN link goes down and then recovers. Basically I lose some kind of routing and it won't recover without a reboot of pfsense.

I have confirmed that DNS actually starts working again, but hitting web ports (port 80, etc) on the internet won't go through. This may or may not be the same issue, but the next time it happens, I'll try to gather the output requested above and submit here if it seems similar.

Jim Pingle wrote:

One thing I did notice in your original description is that the network config is invalid. You can't have LAN as 10.20.30.1/24 and WAN as 10.20.30.254/24 -- two interfaces in the same subnet will mess with the link routes and could cause problems like you see.

Oh well... The 10.20.30.254/24 is another box, with pfSense "WAN" connected to the LAN port of the firewall in front of it...