Project

General

Profile

Actions

Bug #11494

closed

Wireguard interface sends ICMP Redirect when routing between two peers

Added by Blaine Palmer about 3 years ago. Updated 11 months ago.

Status:
Rejected
Priority:
Low
Assignee:
-
Category:
WireGuard
Target version:
-
Start date:
02/21/2021
Due date:
% Done:

0%

Estimated time:
Plus Target Version:
Release Notes:
Force Exclusion
Affected Version:
2.5.0
Affected Architecture:
All

Description

When PFSense is used to route traffic between two WireGuard peers, it send ICMP Redirect when both peers are on the same WireGuard interface.

Example topology

Home PFSense
- LAN 172.20.20.0/24
- WG0 172.20.50.10/32

Concentrator PFSense
- WG0 172.20.50.1/24

MacOS Client
- WG0 172.20.50.20/32

Abridge Concentrator WG Config

  1. Peer: ConcentratorToHome
    [Peer]
    AllowedIPs = 172.20.20.0/24, 172.20.50.10/32
  1. Peer: ConcentratorToMacbook
    [Peer]
    AllowedIPs = 172.20.50.20/32

In this scenario, with traffic being allowed from 172.20.50.20 (MacBook Source) -> 172.20.50.1 (Concentrator router #1) -> 172.20.50.10 (Home router #2) -> 172.20.20.0/24 (destination) you will see something like the following

64 bytes from 172.20.20.1: icmp_seq=0 ttl=63 time=68.083 ms
92 bytes from 172.20.50.1: Redirect Host(New addr: 172.20.20.1)

I believe this is occurring because from Concentrator's perspective, the source and destination all route through the same interface. In most situations I think this behavior is correct, however in the case of WireGuard where multiple peers exist on the same interface I think ICMP Redirect should not be sent.

Actions #1

Updated by Blaine Palmer about 3 years ago

Just for reference, it appears a similar issue was observed early in WireGuard's original development.

https://git.zx2c4.com/wireguard-monolithic-historical/commit/?id=1e96d7f29551309f1ab5480e39dcc6124ea89aa0

Similarly, it appears this similar code made it into the Linux Kernel implementation as well

https://git.zx2c4.com/wireguard-linux/tree/drivers/net/wireguard/device.c
https://github.com/torvalds/linux/blob/master/drivers/net/wireguard/device.c

if (dev_v4) {
/* At some point we might put this check near the ip_rt_send_ * redirect call of ip_forward in net/ipv4/ip_forward.c, similar * to the current secpath check.
*/
IN_DEV_CONF_SET(dev_v4, SEND_REDIRECTS, false);
IPV4_DEVCONF_ALL(dev_net(dev), SEND_REDIRECTS) = false;
}
Actions #2

Updated by Blaine Palmer about 3 years ago

It would appear this may need to be corrected in the FreeBSD upstream.

Possibly relevant:
https://github.com/freebsd/freebsd-src/blob/main/sys/netinet/ip_input.c
Line #119 (sysctl check)
Line #1040 (conditional logic)

https://github.com/freebsd/freebsd-src/blob/main/sys/netinet/ip_var.h
Line 185 VNET_DECLARE(int, ipsendredirects);
Line 201 #define V_ipsendredirects VNET

I tried to poke around and look for other interfaces that were selectively disabling forward/redirect or other sysctl, but am not familiar enough with the code.
Thank you!

Actions #3

Updated by Blaine Palmer about 3 years ago

One last interesting tidbit, similar assumptions causing issues with p2p interfaces in ipv6 which caused issues for WG pre-kernel implementation.

https://github.com/freebsd/freebsd-src/commit/1e9b8db9b254b98ae065889f8c33b0dcc18138a1

Actions #4

Updated by Jim Pingle about 3 years ago

  • Priority changed from Normal to Low
  • Target version changed from 2.5.1 to CE-Next

This is likely a (mostly?) harmless side effect of how the routes in the routing table are added for WireGuard. Because they are added to the interface directly and not to a remote peer (due to how WireGuard operates), redirects are triggered.

This isn't limited to WireGuard. There is an old IPsec workaround to nudge firewall traffic into a policy-based IPsec tunnel which involves added a route using the firewall itself as a gateway. When using that kind of route, hosts on LAN also get the redirects.

If you know you have no need for ICMP redirects on any interface, you can also disable them completely by setting system tunables for:

net.inet.ip.redirect=0
net.inet6.ip6.redirect=0
Actions #5

Updated by Blaine Palmer about 3 years ago

I've disabled redirect via the sysctl/tunable as suggested already.

Just to clarify this is for every incoming packet that is forwarded by FreeBSD/pfSense from one peer on a WireGuard interface to another peer on the same WireGuard interface.

I think the concern would be the overhead this would add unnecessarily. If I understand the FreeBSD ip_forward function properly, it will be sending a redirect for every non-fragmented packet forwarded.

I will test this and get back to you shortly with some packet statistics to confirm if this assertion is correct.

Actions #6

Updated by Jim Pingle about 3 years ago

Another workaround is to do one peer per tunnel and a dynamic routing protocol like BGP, or routes using the remote peer tunnel address as the gateway (fill in the Peer WireGuard Address appropriately, leave Allowed IPs blank, add your own manual static routes)

If it's the routes targeting the interface itself to blame, that should take it out of the equation.

Actions #7

Updated by Blaine Palmer about 3 years ago

I was able to confirm that there does not appear to be any rate limiting, the overhead isn't terrible though as the ICMP packets are so small. Getting statistics was a little more difficult on this new MacBook however and your proposed solution is the best I believe. This simplifies firewall rules and routing semantics.

Thank you

Actions #8

Updated by Jim Pingle almost 3 years ago

  • Target version changed from CE-Next to Future
Actions #9

Updated by Christian McDonald about 2 years ago

  • Status changed from New to Rejected
  • Target version deleted (Future)
  • Release Notes changed from Default to Force Exclusion

Unable to replicate.

We can revisit if someone can demonstrate that this issue is still valid.

Actions #10

Updated by → luckman212 almost 2 years ago

Christian McDonald Just for the record, I'm hitting this exact issue right now on current 22.05 snaps, with WG 0.1.6_1 package and 2 configured remote peers connected. It was easy to reproduce. Found a fairly recent forum thread with the same issue too.

Is the net.inet.ip.redirect=0 tunable still the best workaround we have here?

Actions #11

Updated by Andrei Caba 11 months ago

Christian McDonald wrote in #note-9:

Unable to replicate.

We can revisit if someone can demonstrate that this issue is still valid.

This issue is still valid. We just installed Wireguard on pfsense 2 days ago and we have the same behaviour.
The "workaround" "net.inet.ip.redirect=0" is not really acceptable, as we want to use this tunable enabled (for other purposes).
Can this be fixed in some other way?...

PS - to make it easier for anyone else to replicate:
we used this tutorial:
https://www.wundertech.net/how-to-set-up-wireguard-on-pfsense/
We have pfSense Community Edition (virtual machine, not physical).

Actions

Also available in: Atom PDF