Bug #13753
open
Gateway groups stop sending traffic if they contain wireguard tunnels
0%
Description
I have a dual-isp setup running on an xg7100. Cox and Starlink. I have been able to configure two wireguard tunnels, one to run on each link, and I've setup static routes so that each wireguard tunnel is using the corresponding link to flow (which has been a challenge. it will sometimes try and ride the other tunnels transit network IP and run one tunnel inside of another).
If I setup a gateway group that puts both tunnels in tier 1 and Starlink drops, no traffic will flow through the gateway group until I go and manually disable the starlink wireguard interface, then it successfully falls back to Cox. I think this has something to do with how wireguard binds to interfaces, which happens behind the scenes as that isn't exposed to the user (you can't configure wireguard to bind to a specific interface)
I can create a script where if that tunnel goes down, it'll turn off the interface, but then the catch 22 becomes "how can I tell when the problem is gone to bring the link back up"?
A thought I had to address this was to somehow isolate wireguard tunnels into jails, and run multiple instances of wireguard, each getting one tunnel because it seems like if one tunnel breaks "all of wireguard breaks with it". This doesn't seem to affect remote access vpns, only site to site traffic. Another thought I had was to run each wireguard instance inside of docker, but having a firewall run docker seems like a step in the wrong direction.
Files
Updated by Dan Tentler over 1 year ago
- File gateway_groups.png gateway_groups.png added
Today, Cox went down. In theory, the gateway group should have automatically switched over to starlink, and the wg_s2s gateway (the wireguard tunnel flowing across cox) shouldn't be up - but what has happened is that wireguard has pushed that tunnel across the starlink connection and default routing went down - I had to manually go edit the gateway group to change starlink to being in tier 1 and cox in tier 2 for the link to come back up.
it seems like using wireguard tunnels in gateway groups if there is more than one wireguard tunnel is broken
Updated by Jeff Kuehl over 1 year ago
In my case I do Load Balancing of Wireguard Tunnels, if I add only Wireguard tunnels it only uses one tunnel.
Secondly, if I replace the Load Balancing gateways with OpenVPN gateway(s) (how it typically is run) as tier 1 and change the 2 Wireguard gateways to Tier 2 it will not load balance and only uses 1 OpenVPN gateway/tunnel.
It is not until I remove both Wireguard gateways from the Gateway Group that immediatly both OpenVPN gateways/tunnels are used.