Project

General

Profile

Actions

Bug #4479

open

Firewall rules won't match GRE interface after applying IPSEC transport encryption on GRE tunnel

Added by Jonathan Black about 9 years ago. Updated over 2 years ago.

Status:
New
Priority:
Normal
Assignee:
Category:
Operating System
Target version:
Start date:
02/27/2015
Due date:
% Done:

0%

Estimated time:
Plus Target Version:
Release Notes:
Affected Version:
All
Affected Architecture:
All

Description

I have an issue with IPSEC where my GRE tunnels work fine until I turn on transport encryption with IPSEC. After IPSEC is enabled, I can ping across the tunnel (I can also ping between the hosts on both ends), but any connections across the tunnel will be blocked by the PFSense router on the far end (It appears that none of my rules match anymore and only the default block rule will match). I have been able to reproduce this bug in a physical and virtual environment (I have it running in Hyper-V and can produce that if you wish). Everything will start working correctly again if I disable IPSEC on both ends of the tunnel.

I've attached the backup files for both R1 and R2 PFsense routers. They are configured just as show in the Network Map attached. The GRE tunnel is not shown on the map. R1's GRE interface is 192.168.112.1/24. R2's GRE interface is 192.168.112.2/24. The 192.168.25.X network is the WAN interfaces on the routers with the 172.X.X.X interfaces are the LAN interfaces. The default password is "pfsense" on these.

I've also attached pictures of my firewall rules (Allow Everything) and then pictures of the log where an RDP connection is being blocked.

Please let me know if there is anything else I can do to help provide you additional information.


Files

config-R1.localdomain-20150227194833.xml (16.5 KB) config-R1.localdomain-20150227194833.xml R1 Config Jonathan Black, 02/27/2015 03:25 PM
config-R2.localdomain-20150227194831.xml (16.5 KB) config-R2.localdomain-20150227194831.xml R2 Config Jonathan Black, 02/27/2015 03:25 PM
Firewall_Log.JPG (64.4 KB) Firewall_Log.JPG Firewall Log Jonathan Black, 02/27/2015 03:25 PM
Firewall_Rules_GRE.JPG (26.5 KB) Firewall_Rules_GRE.JPG Firewall Rules on GRE interface Jonathan Black, 02/27/2015 03:25 PM
Firewall_Rules_IPSEC.JPG (26.7 KB) Firewall_Rules_IPSEC.JPG Firewall Rules on IPSEC interface Jonathan Black, 02/27/2015 03:25 PM
Network_Map.JPG (99.1 KB) Network_Map.JPG Network Map Jonathan Black, 02/27/2015 03:25 PM
floating_rule_to_block_gre_output.png (71.8 KB) floating_rule_to_block_gre_output.png Wagner Sartori Junior, 08/23/2017 07:02 AM

Related issues

Related to Bug #8686: IPsec VTI: Assigned interface firewall rules are never parsedNew07/24/2018

Actions
Related to Todo #12289: Update "IPsec Filter Mode" option values and help text to reflect that VTI mode also helps transport mode (e.g. GRE)ResolvedJim Pingle

Actions
Actions #1

Updated by Chris Buechler over 8 years ago

  • Category changed from IPsec to Operating System
  • Status changed from New to Confirmed
  • Assignee set to Chris Buechler
  • Affected Version changed from 2.2 to 2.2.x

this is an issue, end up with state mismatches. Can work around with floating rules w/sloppy state and any TCP flags.

Needs review on stock FreeBSD 11 to see if it's been fixed yet.

Actions #2

Updated by Chris Buechler over 8 years ago

this doesn't appear to happen on stock FreeBSD 11 unless there's something more to it than a single pass rule with keep state. Need to evaluate further to determine if it's still an issue in 2.3/10-STABLE.

Actions #3

Updated by Jorge Albarenque almost 8 years ago

I can confirm this still happens with both GRE and GIF tunnels over IPsec on pfSense 2.3.1

Actions #4

Updated by Chris Buechler over 7 years ago

  • Assignee deleted (Chris Buechler)
  • Affected Version changed from 2.2.x to All
Actions #5

Updated by Jim Thompson over 7 years ago

  • Assignee set to Jonathan Black

yet another case where we lost track of the bug because Chris just removed himself when he left.

assigned back to original reporter to see if it still occurs. assign to me if still valid on 2.3.2 or later.

Actions #6

Updated by Jorge Albarenque over 7 years ago

I can confirm this still occurs on 2.3.2. Probably worth checking on 2.4 since Chris had mentioned it seemed to be resolved on FreeBSD 11

Actions #7

Updated by Jonathan Black over 7 years ago

Jorge Albarenque wrote:

I can confirm this still occurs on 2.3.2. Probably worth checking on 2.4 since Chris had mentioned it seemed to be resolved on FreeBSD 11

It is broken in 2.4 as well.

Actions #8

Updated by Jonathan Black over 7 years ago

It appears to be worse than before now too.... ICMP doesn't work across the tunnel now either.

Actions #9

Updated by Jim Pingle over 7 years ago

Testing on 2.4 won't be reliable until #6937 is fixed.

Actions #10

Updated by Jonathan Black over 7 years ago

Jim Pingle wrote:

Testing on 2.4 won't be reliable until #6937 is fixed.

Apparently this only affects mobile IPSEC. Is this still applicable?

After some further testing, ICMP does work across the tunnel (Windows Firewall tricked me), but TCP connections are still being blocked (I can see it in the log)

Actions #11

Updated by Phil Lavin about 7 years ago

Has anyone managed to test on 2.4 yet? Experiencing this issue in 2.3 latest.

Actions #12

Updated by Phil Lavin about 7 years ago

Confirmed still not working on 2.4

Actions #13

Updated by Brett Howard about 7 years ago

This affects both GRE over IPSEC transport and IPSEC tunnel mode carrying a GRE

All traffic exiting the GRE tunnel is seen as coming from the host itself and matched by the following rule inserted by filter.inc, which comes way before any user rules:

  1. let out anything from the firewall host itself and decrypted IPsec traffic
    pass out {$log['pass']} inet all keep state allow-opts tracker {$increment_tracker($tracker)} label "let out anything IPv4 from firewall host itself"

It is therefore impossible to filter traffic coming in across the GRE at all and state is not created for the return traffic on the GRE interface.

The 'workaround' of creating a floating rule on GRE with sloppy state, combined with the matching above effectively means no firewall at all on the GRE tunnel and is not an accepable solution!

Tested on the latest 2.4 snapshot:
2.4.0-BETA (amd64)
built on Tue Mar 21 07:14:38 CDT 2017
FreeBSD 11.0-RELEASE-p8

Actions #14

Updated by Jim Thompson about 7 years ago

  • Assignee changed from Jonathan Black to Luiz Souza
Actions #15

Updated by Jorge Albarenque over 6 years ago

I don't see any target version on this bug. Is this being worked on? Any chances this could be fixed for 2.4?

Actions #16

Updated by Wagner Sartori Junior over 6 years ago

I'm on 2.4-RC... if I reset the states, it starts working.

Actions #17

Updated by Jim Pingle over 6 years ago

Wagner Sartori Junior wrote:

I'm on 2.4-RC... if I reset the states, it starts working.

Check your states for traffic matching what should be the GRE outer and inner traffic when it does and does not work. What is the difference?

Actions #18

Updated by Wagner Sartori Junior over 6 years ago

  1. Working
    enc0 gre 1.1.1.1 -> 2.2.2.2 MULTIPLE:MULTIPLE
  1. Not Working
    pppoe0 gre 1.1.1.1 -> 2.2.2.2 MULTIPLE:MULTIPLE

I tried to kill only this state, but it also doesn't work. only flushing the whole state table made it work again.

Actions #19

Updated by Jim Pingle over 6 years ago

Add a floating rule to block, quick, in the out direction on that WAN (pppoe0) any GRE traffic, then reboot and see if it makes a difference.

Actions #21

Updated by Jim Pingle over 6 years ago

OK, if that fixed it, then it isn't related to the problem originally stated on this ticket, which is for a state issue inside the tunnel.

Actions #22

Updated by Wagner Sartori Junior over 6 years ago

sorry for that than.

Actions #23

Updated by Michael OBrien over 6 years ago

Any chance 2.4.0, with the FreeBSD 11.1 ipsec changes, may resolve this?

Actions #24

Updated by Michael OBrien over 6 years ago

Michael OBrien wrote:

Any chance 2.4.0, with the FreeBSD 11.1 ipsec changes, may resolve this?

Just loaded up 2.4.0 on a few firewalls and tested. Unfortunately, the bug still exists.

Actions #25

Updated by Daniel Berteaud over 6 years ago

I can confirm that 2.4.2_1 is still affected. So for now, its not possible to do site to site IPSec tunnel (except in tunnel mode, which doesn't scale as soon as you have more than two or three networks per site). Any workaround ? For now I'm using OpenVPN, which is far easier to deal with, but is a bottleneck performance wise on high speed links.

Actions #26

Updated by Eric Dombroski about 6 years ago

Can someone confirm whether or not this bug explains the following situation?

I have a GRE tunnel set up between OpenBSD and pfSense, secured via IPSec in transport mode. From the remote OpenBSD system, I can ping a host in my pfSense "LAN" subnet even though there is no rule allowing that traffic (I've disabled the default any/any). I can also send TCP SYN packets; the replies never get back through, blocked by the default IPv4 block rule.

Running 2.4.2_p1. Is this also an issue with upstream FreeBSD?

Actions #27

Updated by Eric Dombroski about 6 years ago

For what it is worth, I have reproduced this on stock 12-CURRENT.
-Eric

Actions #28

Updated by Daniel Berteaud about 6 years ago

For reference, the upstream bug opened by Eric: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=226411

Actions #29

Updated by Jim Thompson about 6 years ago

Ermal says there is code in Darwin that addresses this.

Actions #30

Updated by Jim Thompson about 6 years ago

  • Target version set to 2.4.4
  • Affected Architecture All added
  • Affected Architecture deleted ()
Actions #31

Updated by Wagner Sartori Junior almost 6 years ago

Does pfSense patch freebsd kernel for some custom/not working on plain kernel? It will take some time until somebody on freebsd actually implement that.

Darwin freebsd kernel probably have this fixed as explained earlier.

He recently said to check the current code and compare:
https://github.com/apple/darwin-xnu/blob/master/bsd/net/pf.c

Actions #32

Updated by Anonymous over 5 years ago

  • Status changed from Confirmed to New
Actions #33

Updated by Anonymous over 5 years ago

  • Status changed from New to 13
Actions #34

Updated by Anonymous over 5 years ago

  • Status changed from 13 to New
Actions #35

Updated by Anonymous over 5 years ago

  • Target version changed from 2.4.4 to 48
Actions #36

Updated by Anonymous over 5 years ago

  • Priority changed from High to Normal
Actions #37

Updated by Wagner Sartori Junior over 5 years ago

Interested to know why do you (or pfSense) think this is not a high priority.

On my point of view (that I know doesn't matter to you or pfSense), this is a huge problem as there's no way to restrict the traffic on the GRE interfaces when running through ipsec.

Actions #38

Updated by Vladyslav Halapsin over 5 years ago

I confirm the problem in the version 2.4.4

Actions #39

Updated by Jim Pingle about 5 years ago

  • Target version changed from 48 to 2.5.0
Actions #40

Updated by Anonymous over 3 years ago

  • Target version changed from 2.5.0 to Future
Actions #41

Updated by Arthur Wiebe over 2 years ago

I stumbled across this issue when deploying pfSense for a wireless carrier integration. We needed to do things like policy routing and so the workaround using floating rules is not an option as it does not match incoming traffic.

As a result we had to use an alternative open source router/firewall platform for the project.

Actions #42

Updated by Jim Pingle over 2 years ago

This is similar, if not identical, to #8686 -- and the same workaround functions for both, it turns out.

You can make this work by going to VPN > IPsec on the Advanced Settings tab, and setting IPsec Filter Mode to filter on VTI assigned interfaces. This will block all tunnel mode traffic, but the states will work correctly in both directions on transport mode GRE interfaces.

I'll update the GUI text to reflect that discovery.

Actions #43

Updated by Jim Pingle over 2 years ago

  • Related to Bug #8686: IPsec VTI: Assigned interface firewall rules are never parsed added
Actions #44

Updated by Jim Pingle over 2 years ago

  • Related to Todo #12289: Update "IPsec Filter Mode" option values and help text to reflect that VTI mode also helps transport mode (e.g. GRE) added
Actions

Also available in: Atom PDF