Project

General

Profile

Actions

Bug #15449

open

Delete disabled gateway prevents static routes being loaded on boot

Added by Larry Fahnoe 17 days ago. Updated about 12 hours ago.

Status:
Confirmed
Priority:
Normal
Assignee:
-
Category:
Routing
Target version:
Start date:
Due date:
% Done:

0%

Estimated time:
Plus Target Version:
24.07
Release Notes:
Default
Affected Version:
Affected Architecture:
amd64

Description

I have a pair of 4200s which were running 23.09.1
Both have an old gateway in a disabled state (see Disabled gateway.png)
There is an IPsec VTI between the 4200s & thus each has a static route to the remote network (Static route.png)

After upgrading to 24.03 I deleted the unused gateway, but upon reboot the static route is not loaded, thus traffic to the remote site doesn't flow. Initially I thought this was due to the state policy change but this is not the case; traffic flows properly as long as the static routes are in place on each end.

This is happening on both systems and is easy for me to replicate: simply roll back to the config with the disabled gateway, reboot (part of the restore) delete the gateway and then reboot. If I leave the disabled gateway in place, the static route is loaded properly upon boot.

The attached screenshots should be self explanatory & the attached system.log covers the boot where the static route was not applied. I have not identified any messages that accompany the problem of the missing route.

--Larry


Files

Disabled gateway.png (100 KB) Disabled gateway.png Shows the old, disabled gateway Larry Fahnoe, 04/29/2024 03:11 PM
Static route.png (47.5 KB) Static route.png Shows the static route Larry Fahnoe, 04/29/2024 03:12 PM
Routes before reboot.png (166 KB) Routes before reboot.png Highlighting the static route before reboot Larry Fahnoe, 04/29/2024 03:15 PM
Routes after reboot.png (158 KB) Routes after reboot.png Showing the route missing Larry Fahnoe, 04/29/2024 03:15 PM
system.log (32 KB) system.log Log of "after reboot" when route went missing Larry Fahnoe, 04/29/2024 03:28 PM
Actions #1

Updated by Jim Pingle 17 days ago

  • Project changed from pfSense Plus to pfSense
  • Category changed from Configuration Backend to Routing
  • Target version set to 2.8.0
  • Affected Plus Version deleted (24.03)
  • Plus Target Version set to 24.07
Actions #2

Updated by Larry Fahnoe 17 days ago

Additional information.

The gateway that is disabled was originally used with a fiber provider's ONT/router which was operating as a router, thus the WAN int and WANGW gateway had static RFC-1918 addressing. When I reconfigured the ONT/router to operate as a bridge, I switched the WAN int from Static IPv4 to DHCP, which caused pfSense to create a new gateway WAN_DHCP which is what is currently being used. Thinking I might need to go back to using the router I disabled rather than deleted the WANGW.

Now, even though the WANGW is disabled, I'm noting that there is a dpinger trying to monitor an IP that is no longer reachable, so I go to edit WANGW to disable gateway monitoring. System won't allow that because the static IP is no longer within the interface's network, so I blanked out the IP and disabled gateway monitoring.

Upon rebooting, the static route is now gone once again.

Actions #3

Updated by Azamat Khakimyanov 9 days ago

I've tested on 23.09.1
- I've added disabled WAN gateway which is not in the same subnet as a real WAN subnet is
- then I added VTi IPsec tunnel, assigned it and enabled.
- then I added static route via this IPsec VTi
Finally I've updated 23.09.1 up to latest 24.03 and I wasn't able to reproduce this issue with static route not being loaded after deleting disabled WAN gateway.

Please delete wrong gateway, delete this Routed IPsec tunnel and recreate it from scratch.
If you still have issue with static routes via IPsec VTi, please open a ticket on our support platform: https://portal.netgate.com

Actions #4

Updated by Marcos M 8 days ago

  • Status changed from New to Incomplete
Actions #5

Updated by Larry Fahnoe 8 days ago

So on one of the 4200s running 24.03 I have done the following:

1. Deleted static route to 192.168.5.0/24
2. Deleted ALEX_MPLS IPsec VTI interface assignment
3. Deleted IPsec P1 and P2
4. Rebooted

1. Add IPsec P1
2. Add IPsec P2
3. Apply changes
4. Add interface OPT3 ipsec1 (IPsec VTI: Mpls), enable, rename to ALEX_MPLS
5. Apply changes
6. Add static route to 192.168.5.0/24 via ALEX_MPLS
7. Apply changes
8. Routes are present as expected and can ssh to remote pfSense via IPsec
9. Reboot

ALEX_MPLS interface is up and both sides are pingable
Static route to 192.168.5.0/24 does not show up which is the problem I'm attempting to document

# netstat -in | grep ipsec1
ipsec1    1400 <Link#9>          ipsec1                                387     0     0      117     0     0
ipsec1       - fe80::%ipsec1/64  fe80::92ec:77ff:fe8e:66e4%ipsec1        0     -     -        2     -     -
ipsec1       - 192.168.8.0/30    192.168.8.2                           114     -     -      112     -     -
# 
# netstat -rn4
Routing tables

Internet:
Destination        Gateway            Flags     Netif Expire
default            47.7.216.1         UGS        igc3
47.7.216.0/21      link#4             U          igc3
47.7.222.194       link#6             UHS         lo0
70.59.119.49       47.7.216.1         UGHS       igc3
127.0.0.1          link#6             UH          lo0
192.168.3.0/24     link#3             U          igc2
192.168.3.1        link#6             UHS         lo0
192.168.8.1        link#9             UH       ipsec1
192.168.8.2        link#6             UHS         lo0
192.168.10.1       link#6             UH          lo0
# 
# ping -qc 2 192.168.8.1 
PING 192.168.8.1 (192.168.8.1): 56 data bytes

--- 192.168.8.1 ping statistics ---
2 packets transmitted, 2 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 24.621/24.764/24.907/0.143 ms
# 
# ping -qc 2 192.168.8.2
PING 192.168.8.2 (192.168.8.2): 56 data bytes

--- 192.168.8.2 ping statistics ---
2 packets transmitted, 2 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 0.255/0.289/0.322/0.033 ms

This configuration was working with 23.09.1. I will attempt to open a ticket per your request, however I only have TAC-lite.

--Larry

Actions #6

Updated by Lev Prokofev 8 days ago

Ticket for reference #2703470963 the SOs and steps included.

Actions #7

Updated by Lev Prokofev 8 days ago

I'm not able to replicate it on 24.03.

Actions #8

Updated by Lev Prokofev 7 days ago

I finally replicated the issue by restoring the config from the status output file, the root cause is still unknown however I was able to find a way to fix the behavior:

1) Move the static route to another gateway or delete it
2) Delete the IPsec gateway
3)Re-create IPsec gateway
4)Move/set a static route to the IPsec gateway

Actions #9

Updated by Steve Wheeler about 12 hours ago

  • Status changed from Incomplete to Confirmed
Actions

Also available in: Atom PDF