Project

General

Profile

Actions

Bug #11960

open

Gateway Monitoring Traffic Goes Out Default Gateway

Added by James Blanton 2 months ago. Updated about 1 month ago.

Status:
Feedback
Priority:
Normal
Assignee:
-
Category:
Gateway Monitoring
Target version:
-
Start date:
05/25/2021
Due date:
% Done:

0%

Estimated time:
Plus Target Version:
Release Notes:
Default
Affected Version:
Affected Architecture:

Description

I'm using pfSense Plus 21.02.2 with a SG-3100 and XG-7100 1U. On both systems, I have dual WAN connections with gateway monitoring. I have found that if I lose the non-default gateway WAN, the "monitoring" traffic for that gateway switches over and goes out of the default gateway.

Under normal circumstances, whenever WAN1 is set to monitor 8.8.8.8 and WAN2 is set to monitor 8.8.4.4, the pings for 8.8.8.8 go out of the WAN1 interface and 8.8.4.4 goes out of the WAN2 interface. However, if WAN2 (non-default gateway) goes down, the ping traffic automatically switches to go out of the WAN1 interface. Even after the WAN2 connection is restored (confirmed by the link-state switching to "UP" and the IP address re-populating), the "loss" continues to rise until it reaches 100%. It will STAY at 100% until either the default gateway goes down or dpinger is restarted. If dpinger is restarted, both gateways immediately show as "UP".

I have also tested this in a gateway group set to failover on packet loss or high latency. In this case, if WAN1 is the current gateway and it goes down long enough for WAN2 to become the gateway, then the monitoring traffic for WAN1 will exit the WAN2 interface. Without intervention, WAN1's gateway will remain offline indefinitely. Once dpinger is restarted, both gateways immediately show as back online.

This behavior was not observed in previous versions.

Actions #1

Updated by Jim Pingle 2 months ago

  • Status changed from New to Feedback
  • Priority changed from Very High to Normal

This sounds similar to #11296 or another routing issue that was fixed already -- please re-test on a development snapshot and see if you can replicate the behavior there.

Actions #2

Updated by James Blanton about 1 month ago

Jim, Sorry for the delay but I've been out of the office a good bit the past month.

I've updated the SG-3100 to 21.05 and the issue was still there. I just updated to the latest dev. build (21.09.a.20210621.0100) and the bug still exists there as well. I performed a packet capture on my "WAN1" interface and then unplugged my "WAN2" interface. After WAN2's GW was marked as "offline", I began to see the monitoring pings from the WAN2 address going out of the WAN1 interface.

I've also recreated a failover GW group on "packet loss/latency". Anytime the "non-default" gateway goes offline, the traffic is routed out the default gateway instead of the specified gateway.

Actions #3

Updated by James Blanton about 1 month ago

UPDATE! Bug only exists upon "link down"

SETUP:

  1. Dual WAN connections
  2. GW group configured as
    1. failover on "packet loss or high latency"
    2. WAN1 is tier 1
    3. WAN2 is tier 2
  3. GW group is the default route

FINDINGS:

  1. If you unplug the WAN connection from the router and plug it back in BEFORE pfSense marks it as down, dpinger works as intended
  2. If you unplug the WAN connection from the router and plug it back in AFTER pfSense marks it as down, dpinger will send the pings for that WAN gateway out of the current/active default gateway and the loss will continue until it reaches 100%, even after WAN connectivity is restored. For example, in the case that WAN1 is the active gateway:
    1. If WAN2 is unplugged and goes offline, dpinger will route the pings to the monitor IP from the WAN2 IP through WAN1, and WAN2 will continue until loss is 100%
    2. If WAN1 is unplugged and goes offline, the GW group will switch to WAN2 as the default gateway, and dpinger will route the pings to the monitor IP from the WAN1 IP through WAN2, and WAN1 will continue until loss is 100%
  3. If the ISP and the WAN interface on the router and connected through a switch, then if ISP is disconnected but the link on the router remains UP, then dpinger works as intended. Even if the gateway goes to 100% loss, as soon as the ISP is reconnected, the loss counter starts going down immediately

In summary, if the gateway goes offline AND the interface link is down, then traffic will be routed out of the default gateway.

Actions

Also available in: Atom PDF