Project

General

Profile

Bug #9263

Incorrect ICMP reply when using limiters

Added by Kirill Khazan 15 days ago. Updated 15 days ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
-
Target version:
-
Start date:
01/08/2019
Due date:
% Done:

0%

Estimated time:
Affected Version:
2.4.4
Affected Architecture:

Description

My setup is as follows. pfSense 2.4.4_p2, it maintains a L2TP tunnel to my ISP and all the traffic is configured to go through that interface (WAN_L2TP).

I configured a floating rule (this is the only rule configured, everything else is default):
Action=Pass, Quick=checked, Interface=WAN_L2TP, Direction=out, Address Family=IPV4,Protocol=Any, Gateway=WAN_L2TP_L2TP, In/Out pipes - WanUpQ, WanDownQ. The limiters are simple Tail Drop/FIFO ones, but I also tried FQ_CODEL limiters, the result is the same. With this setup, everything works as far as I can tell, other than... traceroute and mtr. It looks like this:

$ traceroute -n yahoo.com
traceroute to yahoo.com (72.30.35.10), 30 hops max, 60 byte packets
1 192.168.1.3 0.544 ms 0.510 ms 0.485 ms
2 72.30.35.10 9.542 ms 9.715 ms 9.854 ms
3 * * *
4 72.30.35.10 9.645 ms 9.693 ms 9.663 ms
5 72.30.35.10 66.242 ms 69.697 ms 66.402 ms
6 72.30.35.10 67.251 ms 67.036 ms 67.506 ms
7 72.30.35.10 75.363 ms 72.511 ms 86.762 ms
8 72.30.35.10 76.549 ms 76.740 ms 153.401 ms
9 72.30.35.10 153.355 ms 161.405 ms 161.197 ms
10 72.30.35.10 169.345 ms 158.462 ms 158.295 ms
11 72.30.35.10 166.803 ms 164.302 ms 163.989 ms
12 72.30.35.10 171.817 ms 176.850 ms 172.274 ms
13 72.30.35.10 165.496 ms 166.961 ms 181.393 ms
14 72.30.35.10 172.873 ms 170.853 ms 172.007 ms
$ mtr google.com
Host Loss% Snt Last Avg Best Wrst StDev
1. 192.168.1.3 0.0% 4 0.2 0.3 0.2 0.4 0.0
2. 72.30.35.10 0.0% 4 9.0 10.8 9.0 13.3 1.6

As you can see, traceroute shows the same ip (which is the target host's ip) for every hop, but the timings are different. mtr is broken even worse, it only shows my router and then (I guess) the second hop labeled with the target's ip. Further investigating the issue by issuing pings with specific TTL values:
$ ping -t 1 google.com
PING google.com (172.217.21.238) 56(84) bytes of data.
From 192.168.1.3 (192.168.1.3) icmp_seq=1 Time to live exceeded

$ ping -t 2 google.com
PING google.com (74.125.140.101) 56(84) bytes of data.
From wq-in-f101.1e100.net (74.125.140.101) icmp_seq=1 Time to live exceeded
  1. etc, until hop 9

If I set out the "out" pipe in the rule to "none" and leave only the "in" pipe filled with WanUpQ, the problem disappears. If I configure the limiter on LAN or WAN interface, the problem disappears.

Also available in: Atom PDF