Project

General

Profile

Actions

Bug #3666

closed

PMTUD is broken for NATed traffic

Added by Chris Buechler over 7 years ago. Updated almost 7 years ago.

Status:
Resolved
Priority:
High
Assignee:
Ermal Luçi
Category:
Operating System
Target version:
Start date:
05/17/2014
Due date:
% Done:

0%

Estimated time:
Plus Target Version:
Release Notes:
Affected Version:
2.2
Affected Architecture:

Description

Where you have an interface on a system with a lower MTU than other interfaces, send traffic larger than the MTU of the egress interface, and have pf enabled, you end up with a PMTUD black hole.

For example, simple LAN and WAN setup:
- set LAN to 1500 MTU and WAN to 1000
- send something from a host on LAN destined to something on WAN that's bigger than WAN's MTU with DF set, such as with hping:
hping3 -y -d 1400 -2 -p 12345 $dest_IP

where 12345 is the port and $dest_IP is an IP or hostname to send the traffic.

With pf enabled, the packet just disappears, the client gets nothing back. Disable pf, and you get back the appropriate "frag needed, DF set" ICMP error.

The above description and symptoms are specific to 2.2/10-STABLE, as of the most recent snapshot available at the time of this writing.


Files

broken-rules.debug (6.43 KB) broken-rules.debug Chris Buechler, 06/11/2014 05:59 AM
broken-pfctl-vvsr.txt (19.2 KB) broken-pfctl-vvsr.txt Chris Buechler, 06/11/2014 05:59 AM
broken-pfctl-vvsn.txt (2.09 KB) broken-pfctl-vvsn.txt Chris Buechler, 06/11/2014 05:59 AM
Actions #1

Updated by Ermal Luçi over 7 years ago

  • Status changed from New to Feedback

Patch put in to try to handle this case.

For record this is happening due to NAT being applied on packets and the generated ICMP is targeted to the pfSense machine itself.

Actions #2

Updated by Chris Buechler over 7 years ago

  • Subject changed from enabling pf breaks PMTUD to PMTUD is broken for NATed traffic
  • Status changed from Feedback to New

no change. I did confirm it's specific to NATed traffic and updated subject accordingly. Send any packet > egress interface's MTU with pf enabled where it's NATing the traffic, and you get no reply (and it doesn't leave any other interface on the box either). Turn off NAT, or disable pf, and you get the appropriate dest unreachable, frag needed reply.

Actions #3

Updated by Chris Buechler over 7 years ago

Additional data point. This seemingly isn't an issue in stock FreeBSD 10-STABLE. One I had handy:

FreeBSD m6600-vbox-10stable 10.0-STABLE FreeBSD 10.0-STABLE #0 r265018: Sun Apr 27 19:01:41 UTC 2014     root@grind.freebsd.org:/usr/obj/usr/src/sys/GENERIC  amd64

Same scenario with NAT, that system works fine. Seems a regression related to changes we make.

Actions #4

Updated by Ermal Luçi over 7 years ago

You used the same ruleset on stock FreeBSD as pfSense?

Actions #5

Updated by Chris Buechler over 7 years ago

not identical, no. Had the same basic components - scrub all, pass all, nat on. I can throw the completely identical ruleset on there if that'd help. Can't reach that system at this instant to try it.

Actions #6

Updated by Chris Buechler over 7 years ago

I think you're on to something there. This:

scrub all 

nat on em0 from 192.168.0.0/16 to any -> (em0)

pass in all 
pass out all 

seems to work fine.

The attached, a pretty stock allow all ruleset for what we generate, doesn't work. I haven't tried to narrow it down any further, suggestions on likely culprits?

Actions #7

Updated by Jim Thompson over 7 years ago

  • Assignee set to Ermal Luçi
Actions #8

Updated by Ermal Luçi about 7 years ago

  • Status changed from New to Feedback

I think the sysctl that was activated should fix this.

Actions #9

Updated by Ermal Luçi about 7 years ago

  • % Done changed from 0 to 100
Actions #10

Updated by Chris Buechler about 7 years ago

  • Status changed from Feedback to New

no change in behavior

Actions #11

Updated by Ermal Luçi about 7 years ago

Actually try out a next coming snapshot i found a quick hack that will help even for icmp case.

Actions #12

Updated by Ermal Luçi about 7 years ago

  • Status changed from New to Feedback

Did you try to send an tcp/udp packet rather than icmp one?

Actions #13

Updated by Chris Buechler almost 7 years ago

  • Status changed from Feedback to Confirmed

no change. Ermal, msg me and we can both take a look at my test setup.

Actions #14

Updated by Chris Buechler almost 7 years ago

Ermal - no change with the kernel you built. I have a test setup up now that you can reach. /msg me for info.

Actions #15

Updated by Ermal Luçi almost 7 years ago

  • Status changed from Confirmed to Feedback
Actions #16

Updated by Ermal Luçi almost 7 years ago

Teh reply from interface was not being set properly.
Works for me now.

Actions #17

Updated by Chris Buechler almost 7 years ago

  • Status changed from Feedback to Confirmed
  • % Done changed from 100 to 0

no change. Test setup on dev ESX is fully in place now, info on chaos wiki.

Actions #18

Updated by Chris Buechler almost 7 years ago

  • Status changed from Confirmed to Resolved

scratch that, the test box wasn't rebooted post-gitsync and gitsync doesn't apply the relevant change on the fly. This works now

Actions

Also available in: Atom PDF