Project

General

Profile

Actions
Copy link

Regression #14039

open

Limiters do not limit upload traffic passed by policy-routing rules

Added by Marcos M 3 months ago. Updated 2 months ago.

Status:
Confirmed
Priority:
Normal
Assignee:
-
Category:
Traffic Shaper (Limiters)
Target version:
-
Start date:
Due date:
% Done:

0%

Estimated time:
Plus Target Version:
Release Notes:
Default
Affected Version:
2.7.0
Affected Architecture:

Description

Upload traffic is not limited if the rule passing the traffic uses route-to. This last worked in pfSense+ 22.01 and pfSense CE 2.6.0. See https://redmine.pfsense.org/issues/13026#note-15:

Essentially what happens is that we have two states:

all tcp 10.0.2.1:5201 <- 192.168.1.100:44607       ESTABLISHED:ESTABLISHED
   [2078351244 + 3221291264] wscale 6  [1276678361 + 2419064832] wscale 6
   age 00:00:03, expires in 24:00:00, 122313:60993 pkts, 183465201:3171644 bytes, rule 81
   id: a5627f6300000000 creatorid: a17f7a2e gateway: 1.0.2.1
   origif: vtnet2
all tcp 1.0.2.78:50878 (192.168.1.100:44607) -> 10.0.2.1:5201       ESTABLISHED:ESTABLISHED
   [1276678361 + 2419064832] wscale 6  [2078351244 + 3221291264] wscale 6
   age 00:00:03, expires in 24:00:00, 122313:60993 pkts, 183465201:3171644 bytes, rule 77
   id: a6627f6300000000 creatorid: a17f7a2e gateway: 1.0.2.1
   origif: vtnet0

The first state is created by the rule with the limiter, but because that rule also does route-to the packet is passed through pf_test() a second time, which creates the second state. That second state is created by a rule which doesn't have the limiter associated, and that means that when it matches the limiter is not applied. It's that second state that ends up matching incoming packets, so the limiter doesn't get applied there.

Files

limiter.png (108 KB) limiter.png Marco Goetze, 03/25/2023 04:53 AM
Actions
Copy link
 #1

Updated by Marcos M 3 months ago

  • Description updated (diff)
Actions
Copy link
 #2

Updated by Marcos M 3 months ago

The issue can be avoided by creating a floating rule that applies the upload limiter.

Without the floating rule, the download speed is limited and the upload speed is unlimited:

all icmp 1.0.0.1:1 <- 10.0.5.50:1       0:0
   age 00:00:07, expires in 00:00:09, 7:7 pkts, 420:420 bytes, rule 705
   id: 2734f26400000000 creatorid: 4da82510 gateway: 192.168.100.1
   origif: vmx0.5
all icmp 192.168.100.11:36640 (10.0.5.50:1) -> 1.0.0.1:36640       0:0
   age 00:00:07, expires in 00:00:09, 7:7 pkts, 420:420 bytes, rule 155
   id: 2834f26400000000 creatorid: 4da82510 gateway: 192.168.100.1
   origif: vmx0.99

@705 pass in quick on vmx0.5 route-to (vmx0.99 192.168.100.1) inet proto icmp from 10.0.5.0/24 to ! <a_Local4:13> keep state label "USER_RULE: ping to internet" label "id:1529164111" label "gw:WAN_GWV4" ridentifier 1529164111 tag qos_14039_default
  [ Evaluations: 5799      Packets: 1188      Bytes: 70392       States: 1     ]
  [ Inserted: uid 0 pid 16920 State Creations: 1     ]
  [ Last Active Time: Sun Feb 26 21:45:59 2023 ]
@155 pass out route-to (vmx0.99 192.168.100.1) inet from 192.168.100.11 to ! 192.168.100.0/24 flags S/SA keep state allow-opts label "let out anything from firewall host itself" ridentifier 1000015261
  [ Evaluations: 313552    Packets: 18578109  Bytes: 942851185   States: 64    ]
  [ Inserted: uid 0 pid 16920 State Creations: 65    ]
  [ Last Active Time: Sun Feb 26 21:46:16 2023 ]

With the floating rule, both the download and upload speeds are correctly limited:

all icmp 1.0.0.1:1 <- 10.0.5.50:1       0:0
   age 00:00:02, expires in 00:00:10, 3:3 pkts, 180:180 bytes, rule 706
   id: 0b2ef26400000000 creatorid: 4da82510 gateway: 192.168.100.1
   origif: vmx0.5
all icmp 192.168.100.11:55598 (10.0.5.50:1) -> 1.0.0.1:55598       0:0
   age 00:00:02, expires in 00:00:10, 3:3 pkts, 180:180 bytes, rule 281
   id: 0c2ef26400000000 creatorid: 4da82510 gateway: 192.168.100.1
   origif: vmx0.99

@706 pass in quick on vmx0.5 route-to (vmx0.99 192.168.100.1) inet proto icmp from 10.0.5.0/24 to ! <a_Local4:13> keep state label "USER_RULE: ping to internet" label "id:1529164111" label "gw:WAN_GWV4" ridentifier 1529164111 tag qos_14039_default
  [ Evaluations: 5748      Packets: 1120      Bytes: 66312       States: 1     ]
  [ Inserted: uid 0 pid 99814 State Creations: 4     ]
  [ Last Active Time: Sun Feb 26 21:43:13 2023 ]
@281 pass out quick on vmx0.99 route-to (vmx0.99 192.168.100.1) inet from any to ! <a_Local4:13> flags S/SA keep state label "USER_RULE: #14039 default" label "id:1677460989" label "gw:ISP1_STATIC" ridentifier 1677460989 dnqueue 3 tagged qos_14039_default
  [ Evaluations: 46067     Packets: 2124006   Bytes: 2207764474  States: 17    ]
  [ Inserted: uid 0 pid 99814 State Creations: 70    ]
  [ Last Active Time: Sun Feb 26 21:43:48 2023 ]

Without the floating rule and without route-to on the pass in rule, both the download and upload speeds are correctly limited:

all icmp 1.0.0.1:1 <- 10.0.5.50:1       0:0
   age 00:00:01, expires in 00:00:10, 2:2 pkts, 120:120 bytes, rule 705
   id: 0b5cf26400000000 creatorid: 4da82510 gateway: 0.0.0.0
   origif: vmx0.5
all icmp 192.168.100.11:2227 (10.0.5.50:1) -> 1.0.0.1:2227       0:0
   age 00:00:01, expires in 00:00:10, 2:2 pkts, 120:120 bytes, rule 155
   id: 0c5cf26400000000 creatorid: 4da82510 gateway: 192.168.100.1
   origif: vmx0.99

@705 pass in quick on vmx0.5 inet proto icmp from 10.0.5.0/24 to ! <a_Local4:13> keep state label "USER_RULE: ping to internet" label "id:1529164111" ridentifier 1529164111 tag qos_14039_default
  [ Evaluations: 51        Packets: 12        Bytes: 720         States: 0     ]
  [ Inserted: uid 0 pid 99857 State Creations: 2     ]
  [ Last Active Time: Sun Feb 26 22:05:09 2023 ]
@155 pass out route-to (vmx0.99 192.168.100.1) inet from 192.168.100.11 to ! 192.168.100.0/24 flags S/SA keep state allow-opts label "let out anything from firewall host itself" ridentifier 1000015261
  [ Evaluations: 319499    Packets: 18989148  Bytes: 1384500604  States: 94    ]
  [ Inserted: uid 0 pid 99857 State Creations: 123   ]
  [ Last Active Time: Sun Feb 26 22:05:50 2023 ]

For reference, limiters were tested using speedtest.net with IPv4. In this setup, the download limit is applied via a match rule:

match in on { vmx0 vmx0.5 vmx0.50 vmx0.100 } inet from any to ! $a_Local4 tag "qos_default" ridentifier 1652043092 label "USER_RULE: QoS tag default IPv4" label "id:1652043092"
match in on { vmx0 vmx0.5 vmx0.10 vmx0.20 vmx0.50 vmx0.100 } inet from any to any tagged "qos_default" ridentifier 1652459212 dnqueue( 3,6) label "USER_RULE: QoS queue default IPv4" label "id:1652459212"

Actions
Copy link
 #3

Updated by Jose Duarte 2 months ago

I think in general you currently don't need more testers but can at least share that we are quite affected since we have Multi-WAN + GRE tunnels so the limiters + route-to are essencial for all our pfSense and pfSense+ firewalls.
The floating rule workaround is good to know though.

Actions
Copy link
 #4

Updated by Marco Goetze 2 months ago

Tried as Floating Rule but for me same outcome, the Limiter Diagnostic showing no limiter applied.

Actions
Copy link

Also available in: Atom PDF