Bug #11231
closed
OpenVPN tunnel exiting wrong interface
0%
Description
In a multi-WAN environment with multiple OpenVPN tunnels, it seems the tunnels can egress the incorrect WAN interface (not the interface they are manually assigned to). I tested by building two OpenVPN tunnels between FW-A and FW-B.
From client (LHS) to server (RHS)
FW-B WAN1 --> WAN1 FW-A
FW-B WAN2 --> WAN1 FW-A (deliberately WAN1 again on server)
On both firewalls, WAN1 is the system default gateway with a failover to a WAN2.
Without any firewall filter rules both tunnels establish. Can ping, route etc.
I know that for the ISPs used for the WANs there is no uRPF enabled at their end (I used to work there).
Now I decide to strengthen my filter rules to stop incorrect behaviour with my own uRPF setup... with the following two floating tab "Block" rules:
1) Interface: WAN1. Source address: WAN2. Direction "out"
2) Interface: WAN2. Source address: WAN1. Direction "out"
This is clearly to stop source address leakage of WAN1 source IP leaving WAN2, and vice versa.
When enabled, this kills the "FW-B WAN2 --> WAN1 FW-A" tunnel. It seems that that this tunnel wants to source the FW-B WAN2 address (which is good), but route it out the FW-B WAN1 interface as per the default gateway (which is not good, as I told it not to).
To be absolutely clear for this "FW-B WAN2 --> WAN1 FW-A " tunnel I have set "WAN2" as the Interface on the client in the GUI where it says "The interface used by the firewall to originate this OpenVPN client connection" and definitely not "Any", "WAN1" or any gateway group. So correct IP, wrong interface. I double-checked NAT rules and port forwarding, and nothing should be interfering - there's no port forwards, and only the LAN subnet is natted out the WAN, with seperate source NAT rules for WAN1 and WAN2. Should all be fine.
I cannot explain this behaviour other than as a bug.
Seen in production running 2.4.5p1. Replicated in the lab with both 2.4.5p1 and 2.5.0 latest snapshots (7th Jan 2021). I am at your disposal and happy to help. Thank you kindly.
Updated by Gavin Owen over 3 years ago
After wiresharking in the lab, it seems I have miscategorised this issue. When the afforementioned floating tab filter rules are removed, I actually see the "FW-B WAN2 --> WAN1 FW-A" tunnel esablish over the correct FW-B WAN2 interface.
But why then won't it work with this filter rule "Interface: WAN1. Source address: WAN2. Direction "out", Action: block" in place?
OpenVPN says:
"reconnecting; ping-restart Fri Jan 8 0:40:48 2021 (pending)"
and logs say:
"write UDPv4: Permission denied (code=13)"
I can't find anything in the wireshark captures as to why this would be happening. It seems the issue is entirely internal to pfSense, and that OpenVPN is tripping over a perfectly-fine filter rule. I can't edit what's been logged but feel free to update title and description. Based on what I've found, this perhaps should be categorised as an "OpenVPN" rather than "Multi-WAN", but does have elements of both.
Updated by Jim Pingle over 3 years ago
- Status changed from New to Not a Bug
- Target version deleted (
2.5.0)
This isn't a bug, but a side effect of your manual rule causing traffic to not hit a built-in rule that it needs to use for that kind of routing to function.
Outbound traffic must follow the system routing table, which always exits via the default gateway for hosts that don't have another route. Since both of your tunnels connect to the same remote address (itself a suboptimal practice), there is no way that traditional routing could handle that scenario. If the remote addresses were unique, you could add a static route which always forces traffic out the other interface to that specific destination.
There are built-in automatic rules in the ruleset which match traffic attempting to exit via the "wrong" interface (e.g. WAN2 leaving WAN1) and directs traffic out via the correct interface/gateway. Your unnecessary floating rule blocked the traffic before it could reach those rules.
If you have additional questions, please start a thread on the forum.
Updated by Gavin Owen over 3 years ago
Thank you very much for the clarifcation - I will remove the unnecessary filter rules.
Updated by Gavin Owen over 3 years ago
"itself a suboptimal practice" - in most scenarios it would be, but I would have to explain the network topology for you to understand that in this case it works best (there's actually a third site involved, and 5 WAN links in total which have varying degrees of quality and asymmetry... it's fairly complicated ;)
"there is no way that traditional routing could handle that scenario" - would have thought that everything relating to the tunnel establishment from WAN2 would be policy routed out WAN2 and not rely on the default routing table (out WAN1), but see that's not the case. In the purist sense this is a deficiency, but perhaps practicality trumps purity, and things work as is, so will just leave it at that. Thanks again.