Bug #11494
closedWireguard interface sends ICMP Redirect when routing between two peers
0%
Description
When PFSense is used to route traffic between two WireGuard peers, it send ICMP Redirect when both peers are on the same WireGuard interface.
Example topology
Home PFSense
- LAN 172.20.20.0/24
- WG0 172.20.50.10/32
Concentrator PFSense
- WG0 172.20.50.1/24
MacOS Client
- WG0 172.20.50.20/32
Abridge Concentrator WG Config
- Peer: ConcentratorToHome
[Peer]
AllowedIPs = 172.20.20.0/24, 172.20.50.10/32
- Peer: ConcentratorToMacbook
[Peer]
AllowedIPs = 172.20.50.20/32
In this scenario, with traffic being allowed from 172.20.50.20 (MacBook Source) -> 172.20.50.1 (Concentrator router #1) -> 172.20.50.10 (Home router #2) -> 172.20.20.0/24 (destination) you will see something like the following
64 bytes from 172.20.20.1: icmp_seq=0 ttl=63 time=68.083 ms
92 bytes from 172.20.50.1: Redirect Host(New addr: 172.20.20.1)
I believe this is occurring because from Concentrator's perspective, the source and destination all route through the same interface. In most situations I think this behavior is correct, however in the case of WireGuard where multiple peers exist on the same interface I think ICMP Redirect should not be sent.
Updated by Blaine Palmer almost 4 years ago
Just for reference, it appears a similar issue was observed early in WireGuard's original development.
Similarly, it appears this similar code made it into the Linux Kernel implementation as well
https://git.zx2c4.com/wireguard-linux/tree/drivers/net/wireguard/device.c
https://github.com/torvalds/linux/blob/master/drivers/net/wireguard/device.c
if (dev_v4) {
/* At some point we might put this check near the ip_rt_send_
* redirect call of ip_forward in net/ipv4/ip_forward.c, similar
* to the current secpath check.
*/
IN_DEV_CONF_SET(dev_v4, SEND_REDIRECTS, false);
IPV4_DEVCONF_ALL(dev_net(dev), SEND_REDIRECTS) = false;
}
Updated by Blaine Palmer almost 4 years ago
It would appear this may need to be corrected in the FreeBSD upstream.
Possibly relevant:
https://github.com/freebsd/freebsd-src/blob/main/sys/netinet/ip_input.c
Line #119 (sysctl check)
Line #1040 (conditional logic)
https://github.com/freebsd/freebsd-src/blob/main/sys/netinet/ip_var.h
Line 185 VNET_DECLARE(int, ipsendredirects);
Line 201 #define V_ipsendredirects VNET
I tried to poke around and look for other interfaces that were selectively disabling forward/redirect or other sysctl, but am not familiar enough with the code.
Thank you!
Updated by Blaine Palmer almost 4 years ago
One last interesting tidbit, similar assumptions causing issues with p2p interfaces in ipv6 which caused issues for WG pre-kernel implementation.
https://github.com/freebsd/freebsd-src/commit/1e9b8db9b254b98ae065889f8c33b0dcc18138a1
Updated by Jim Pingle almost 4 years ago
- Priority changed from Normal to Low
- Target version changed from 2.5.1 to CE-Next
This is likely a (mostly?) harmless side effect of how the routes in the routing table are added for WireGuard. Because they are added to the interface directly and not to a remote peer (due to how WireGuard operates), redirects are triggered.
This isn't limited to WireGuard. There is an old IPsec workaround to nudge firewall traffic into a policy-based IPsec tunnel which involves added a route using the firewall itself as a gateway. When using that kind of route, hosts on LAN also get the redirects.
If you know you have no need for ICMP redirects on any interface, you can also disable them completely by setting system tunables for:
net.inet.ip.redirect=0 net.inet6.ip6.redirect=0
Updated by Blaine Palmer almost 4 years ago
I've disabled redirect via the sysctl/tunable as suggested already.
Just to clarify this is for every incoming packet that is forwarded by FreeBSD/pfSense from one peer on a WireGuard interface to another peer on the same WireGuard interface.
I think the concern would be the overhead this would add unnecessarily. If I understand the FreeBSD ip_forward function properly, it will be sending a redirect for every non-fragmented packet forwarded.
I will test this and get back to you shortly with some packet statistics to confirm if this assertion is correct.
Updated by Jim Pingle almost 4 years ago
Another workaround is to do one peer per tunnel and a dynamic routing protocol like BGP, or routes using the remote peer tunnel address as the gateway (fill in the Peer WireGuard Address appropriately, leave Allowed IPs blank, add your own manual static routes)
If it's the routes targeting the interface itself to blame, that should take it out of the equation.
Updated by Blaine Palmer almost 4 years ago
I was able to confirm that there does not appear to be any rate limiting, the overhead isn't terrible though as the ICMP packets are so small. Getting statistics was a little more difficult on this new MacBook however and your proposed solution is the best I believe. This simplifies firewall rules and routing semantics.
Thank you
Updated by Jim Pingle almost 4 years ago
- Target version changed from CE-Next to Future
Updated by Christian McDonald almost 3 years ago
- Status changed from New to Rejected
- Target version deleted (
Future) - Release Notes changed from Default to Force Exclusion
Unable to replicate.
We can revisit if someone can demonstrate that this issue is still valid.
Updated by → luckman212 over 2 years ago
Christian McDonald Just for the record, I'm hitting this exact issue right now on current 22.05 snaps, with WG 0.1.6_1 package and 2 configured remote peers connected. It was easy to reproduce. Found a fairly recent forum thread with the same issue too.
Is the net.inet.ip.redirect=0
tunable still the best workaround we have here?
Updated by Andrei Caba over 1 year ago
Christian McDonald wrote in #note-9:
Unable to replicate.
We can revisit if someone can demonstrate that this issue is still valid.
This issue is still valid. We just installed Wireguard on pfsense 2 days ago and we have the same behaviour.
The "workaround" "net.inet.ip.redirect=0" is not really acceptable, as we want to use this tunable enabled (for other purposes).
Can this be fixed in some other way?...
PS - to make it easier for anyone else to replicate:
we used this tutorial:
https://www.wundertech.net/how-to-set-up-wireguard-on-pfsense/
We have pfSense Community Edition (virtual machine, not physical).