Project

General

Profile

Bug #7015

IPsec not working behind NAT

Added by Renato Botelho 6 months ago. Updated about 2 months ago.

Status:
Assigned
Priority:
High
Category:
IPsec
Target version:
Start date:
12/16/2016
Due date:
% Done:

100%

Affected version:
2.4
Affected Architecture:

Description

@luiz has the details, looks like a ESP fragment but it creates odd state with unknown IP address like:

enc0 icmp 69.0.0.84:6748 <- 172.27.10.20:6748 0:0

History

#1 Updated by Renato Botelho 6 months ago

  • Priority changed from Normal to High

#2 Updated by Steve Wheeler 5 months ago

Also seeing this after upgrading to 2.4.

Initially unable to ping across the tunnel but a packet capture showed pings leaving over IPSec and replies coming back. The replies were being blocked in the firewall, not matching the state opened. By adding a rule to pass that reply traffic I am able ping but the state created is indeed very weird:

LAN icmp 172.21.16.5:6442 -> 172.27.34.10:6442 0:0 7 / 6 588 B / 504 B
IPsec icmp 172.21.16.5:6442 -> 172.27.34.10:6442 0:0 7 / 0 588 B / 0 B
IPsec icmp 172.27.34.10:6442 -> 69.0.0.84:6442 0:0 6 / 0 504 B / 0 B

Yet the ping replies are correctly routed back to 172.21.16.5.

Also for UDP traffic:

LAN udp 172.21.16.7:5060 -> 172.27.34.10:5060 MULTIPLE:MULTIPLE 904 / 691 593 KiB / 380 KiB
IPsec udp 172.21.16.7:5060 -> 172.27.34.10:5060 SINGLE:NO_TRAFFIC 904 / 0 593 KiB / 0 B
IPsec udp 172.27.34.10:5060 -> 69.96.2.50:5060 NO_TRAFFIC:SINGLE 648 / 0 356 KiB / 0 B

That VoIP works fine though.

For TCP traffic it appears each state only sees one half of the exchange so both block the out-of-state traffic. I had to add floating rules with direction any, flags any, and sloppy states in order to pass it.

Other users on that same IPSec server hitting the same resources are not seeing this problem. My end point is behind NAT so the tunnel uses NAT-T though.

#3 Updated by Vladimir Putin 5 months ago

#4 Updated by Renato Botelho 5 months ago

Vladimir Putin wrote:

Could it be related to https://redmine.pfsense.org/issues/6937 ?

I believe not. We just found out an evidence this issue only happens when WAN interface is behind NAT

#5 Updated by Renato Botelho 4 months ago

  • Subject changed from IPsec not working with specific ISP to IPsec not working behind NAT

#6 Updated by Luiz Otavio O Souza 4 months ago

  • Status changed from Confirmed to Feedback
  • % Done changed from 0 to 100

Fixed in latest update.

#7 Updated by Luiz Otavio O Souza 4 months ago

  • Status changed from Feedback to Resolved

Fixed (also confirmed by JimP on #6937 which was caused by the same issue).

#8 Updated by David Myers 3 months ago

I’m testing routing all IPv4 and IPv6 LAN traffic through a remote VPN server and am having issues with IPv6 that might be related to this issue.

For testing I have pfSense 2.4 Beta in a VirtualBox. The LAN interface is bridged through the Ethenet adapter with static IPv4 and IPv6 addresses. The WAN interface is NAT-ed so as to appear on a different network and only has an IPv4 address. So outgoing IPv4 traffic from this VM is NAT-ed twice, first through VirtualBox then through my real pfSense box.

I’ve set up an IKEv2 Phase 1 tunnel over IPv4, and have IPv4 and IPv6 Phase 2 tunnels. IPv4 seems to be working fine with no additional firewall rules as long as I use MSS clamping to 1400 on both sides.

IPv6 is acting strange. When I try to ping from a client system (also in VirtualBox but on a different host) to Google DNS at 2001:4860:4860::8888, it fails:

Apr  4 12:58:45 pfSense filterlog: 7,,,1000104535,enc0,match,block,in,6,0x00,0xdcb13,57,ICMPv6,58,64,2001:4860:4860::8888,2602:304:aa7e:d0a0::201,

That rule is:
block in log inet6 all tracker 1000104535 label "Default deny rule IPv6”

In the state table I see:
em1 ipv6-icmp 2001:4860:4860::8888[1319] <- 2602:304:aa7e:d0a0::201[1319]       NO_TRAFFIC:NO_TRAFFIC
enc0 ipv6-icmp 2602:304:aa7e:d0a0::201[1319] -> 2001:4860:4860::8888[1319]       NO_TRAFFIC:NO_TRAFFIC

Running “curl ipv6.icanhazip.com” to try to determine the public IPv6 address of the remote VPN server results in several log entries like this:
Apr  4 13:07:59 pfSense filterlog: 7,,,1000104535,enc0,match,block,in,6,0x00,0x5622e,52,TCP,6,40,2604:7780:200:305:f816:3eff:fe6e:69e9,2602:304:aa7e:d0a0::201,80,38762,0,SA,1568736757,1810540167,28560,,mss;sackOK;TS;nop;wscale

These are, I think, the related states:

em1 tcp 2604:7780:200:305:f816:3eff:fe6e:69e9[80] <- 2602:304:aa7e:d0a0::201[38762]       CLOSED:SYN_SENT
enc0 tcp 2602:304:aa7e:d0a0::201[38762] -> 2604:7780:200:305:f816:3eff:fe6e:69e9[80]       SYN_SENT:CLOSED

I would have expected this traffic to be passed by the default LAN IPv6 to any rule.

Adding the following floating rule makes the above curl command work, but not consistently:

pass  on {  enc0  } inet6 proto tcp  from any to any tracker 1491081937 flags any keep state ( sloppy  )  label “USER_RULE" 

#9 Updated by Jim Pingle 3 months ago

  • Status changed from Resolved to Assigned

#10 Updated by David Myers 2 months ago

Using the setup described above I’ve also been having issues when trying to use IPsec Transport mode with either a GRE or GIF. (Perhaps what I’m seeing is really related to https://redmine.pfsense.org/issues/4479, I’m not sure).

The remote system is a Linux virtual server running strongSwan and either a GRE-to-GRE or GIF-to-IPIP tunnel. The remote inner tunnel address is 10.19.49.2. A test client on my LAN is at 192.168.1.201.

For either GRE or GIF I’m using manual (AON) NAT to avoid the automatically created outbound NAT rules for the GRE or GIF (WAN still uses NAT, though). The only rule I’m adding is on LAN to policy route the test client through the GRE or GIF gateway. So no added rules on the GIF/GRE, IPsec, WAN, or Floating tabs (and no RFC 1918 or bogons rules either).

When using a GIF, dpinger can happily ping the other side and I can see the traffic counters increment on the other side. If I try to ping from the remote VPN server to the LAN client the ping fails as expected, but apparently only if there are no other states for that client. If I cause TCP traffic from the LAN client, the ping from the remote side starts working and one of the created states is reversed:

em1 tcp 64.182.208.184:80 <- 192.168.1.201:37590       CLOSED:SYN_SENT
gif0 tcp 192.168.1.201:37590 -> 64.182.208.184:80       SYN_SENT:CLOSED
em1 icmp 10.19.49.2:10733 -> 192.168.1.201:10733       0:0
gif0 icmp 192.168.1.201:10733 -> 10.19.49.2:10733       0:0

Nothing appears in the firewall logs.

When using a GRE, I can’t even ping the remote side. I get the following error and the GRE interface outbound error counter increments:

PING 10.19.49.2 (10.19.49.2): 56 data bytes
ping: sendto: Permission denied

If I drop IPsec the error goes away, though the pings are unanswered because there are no rules on the remote side to allow a tunnel to come in directly from the WAN.

#11 Updated by David Myers 2 months ago

As of 2.4.0.b.20170421.0857 I'm getting the same ping errors with a GIF. dpinger's attempts to ping do create a state:

gif0 icmp 10.19.49.1:20726 -> 10.19.49.2:20726       0:0

#12 Updated by David Myers about 2 months ago

The problem I'm seeing with a GIF might be a command ordering or race condition issue.

Using 20170428 I set up a new IKEv2 Phase 1, Phase 2 Transport, GIF, and GIF Interface and pings were OK. I could also route LAN traffic through the GIF with a policy route. Then I rebooted and pings failed with the "Permission denied" error above.

#13 Updated by David Myers about 2 months ago

Me again. Here's some more state strangeness.

Today the GIF tunnel is working and I can route traffic from my test client to the Internet, but file transfers stall after about 30 seconds. Here are the states when I do a wget of a large file:

em1 tcp 192.101.102.2:80 <- 192.168.1.201:40950       ESTABLISHED:ESTABLISHED
gif0 tcp 192.168.1.201:40950 -> 192.101.102.2:80       SYN_SENT:CLOSED

The state on the GIF should presumably be ESTABLISHED:ESTABLISHED like the LAN state. After this state disappears the file transfer stops until wget tries again with a new connection.

Also available in: Atom PDF