Project

General

Profile

Actions

Bug #14854

closed

Packets are passed through dummynet twice when using ``route-to`` leading to half the expected bandwidth

Added by nasir ahmed 7 months ago. Updated about 1 month ago.

Status:
Resolved
Priority:
Normal
Category:
Traffic Shaper (Limiters)
Target version:
Start date:
Due date:
% Done:

0%

Estimated time:
Plus Target Version:
24.03
Release Notes:
Default
Affected Version:
2.7.0
Affected Architecture:
amd64

Description

When using a traffic shaper limiter to set bandwidth to say 10mbps in the download using any scheduler, if the gateway is left to default the limiter works as expected but if a specific gateway or a gateway group is specified the limiter works as if it was set to 5mbps.
This was verified on a fresh installation of pfSense 2.7.0 using two wan links and one lan. pfSense 2.6.x works fine.

Actions #1

Updated by nasir ahmed 7 months ago

from: cat /tmp/rules.debug
No gateway specified

anchor "userrules/*"
pass in quick on $LAN inet from 192.168.1.0/24 to any ridentifier 1696786360 keep state dnpipe ( 2,1) label "USER_RULE: Default allow LAN to any rule" label "id:1696786360"

policy routing WAN gateway specified results in bandwidth halved.

anchor "userrules/*"
pass in quick on $LAN $GWWAN_DHCP inet from 192.168.1.0/24 to any ridentifier 1696786360 keep state dnpipe ( 2,1) label "USER_RULE: Default allow LAN to any rule" label "id:1696786360" label "gw:WAN_DHCP"

maybe the packet is sent twice to the pipe!!!

Actions #2

Updated by Lukáš Mojžíš 7 months ago

I've just registered to report this. This affects me too.
The situation can only be mitigated by setting gateway to default.

Actions #3

Updated by dylan mendez 7 months ago

Unable to replicate with the following setup

1 WAN - 1 LAN
pfSense CE 2.7.0 on a VM
Ubuntu Desktop client

Steps taken:

1) System - Routing - Default Gateway set to Automatic
2) Created 2 10mbps Limiters using Worst Case Weighted fair Queueing
3) Modified LAN to Any default rule, added the In/Out pipes
4) Ran Speedtest to Internet - Getting stable 10/10Mbps
5) Changed Default Gateway to WAN_DHCP
6) Cleared States
7) Tried speedtest again, still getting 10/10Mbps

Can you please specify what settings you're using for the limiters?

I'll setup a secondary WAN and try again.

Actions #4

Updated by Lukáš Mojžíš 7 months ago

I am using default "new limiter" UploadLimit and speed limit in bits/s (16*1024*1024)
I am using default "new limiter" DownloadLimit and speed limit in bits/s (16*1024*1024)
I set rule to use a In pipe UploadLimit/ Out pipe DownloadLimit and gateway Gateway

Speed via mobile speedtest.net is 8.0 Mb/s down and 16.0 Mb/s up.

My current theory is that the issue is caused by gateway IP in 10.0.0.0/8 and it somehow lists each packet twice, although I currently have no way of verifying that.

Actions #5

Updated by aleksei prokofiev 7 months ago

I've tested on
2.7.0-RELEASE (amd64)
built on Wed Jun 28 03:53:34 UTC 2023
FreeBSD 14.0-CURRENT

It is not a bug, I suspect it is incorrect setup.
You should use floating rules when setting up limmiters with multi-wan. It is documented.
https://docs.netgate.com/pfsense/en/latest/trafficshaper/limiters.html#limiters-and-multi-wan
When using limiters with Multi-WAN, limits for non-default gateways must be applied using floating rules set for the out direction and configured with the appropriate gateway.

Without floating rules, I got the same result as you, the download speed is lower than the limiter set. With floating rules everything works as expected.

Actions #6

Updated by Jose Duarte 3 months ago

I wouldn't say it's incorrect setup, using limiters on multi-wan setup has been working on pfSense for over 8 years. It became an issue starting from 2.7.0, same time as the upload not being limited issue https://redmine.pfsense.org/issues/14039

Same problem here on 2.7.2. Upload limiter issue is fixed but the download speed is being halved as described. Removing the Gateway fixes it.

Actions #7

Updated by Sav Snip 3 months ago

I am encountering the same issue in a multi-WAN setup. Although the upload problem (https://redmine.pfsense.org/issues/14039) has been resolved since the update, the download speed is halved when using limiters. The issue disappears when the default gateway is selected under firewall rules.

Actions #8

Updated by Marco Goetze about 2 months ago

Greetings,

The current state of the Multi-WAN limiter functionality has been unfortunately problematic for quite some time now. The commonly recommended approach of implementing tagged floating rules as a method of handling this issue, while popular, should not be considered a standard solution or fix due to its introduction of unnecessary complexity. Therefore, I must express my disagreement with the viewpoint that the issue stems from an "incorrect setup" rather than being a bug.

Historical Context

Version 22.01 represented the last stable release for Multi-WAN Limiter functionality on the interface. With the introduction of version 22.05, we faced an issue affecting limiter upload traffic on the interface, which was not limited at all (See https://redmine.pfsense.org/issues/14039). Initial instances revealed that the primary rules for Multi-WAN gateway limiters failed to be recognized, prompting the use of Floating Rule tagging as a temporary fix. An attempt to rectify this issue was made in the version 23.09 update, which only succeeded in partially addressing the problem, resulting in limiters functioning in an inconsistent "Half duplex" mode.
The workaround proposed post-version 22.01—applying limiters via floating rules—was always seen as a clear "workaround until fixed".

Current State
It is not possible to use limiters on Multi-WAN setups in the interface rules, which shouldn't be an issue, and worked as Jose stated since many many years. However - basically it is working, but the rates are halved only on the download side. Hopefully, this is something easy to solve because the basic functionality as it was in version 22.01 is back and working; it is just that the rates in one direction are incorrect. The current easiest workaround would be to double the limiter MBit pool. So if I want to limit a 100M line, I put the limiter at 200M to achieve a 100M result :)

Should anybody require any additional data or debugging assistance from me, please let me know. This can be quickly done.

Thank you all.

edit: fixed some version no. mixup in text

Actions #9

Updated by Marcos M about 2 months ago

  • Status changed from New to Not a Bug
  • Priority changed from Very High to Normal

Indeed using a download limiter on "pass in route-to" rules results in lower bandwidth limits than what is configured (tested in 24.03). I've confirmed this still works correctly when using floating rules in 24.03. The documentation note referenced in #note-5 has been there for the last 8 years which indicates that floating rules have always been the supported configuration for multi-WAN.

Actions #10

Updated by Marcos M about 2 months ago

  • Status changed from Not a Bug to Confirmed
Actions #11

Updated by Marcos M about 1 month ago

  • Subject changed from bandwidth is halved when using a limiter on a multi-homed setup when using policy routing in firewall rule to packets are passed through dummynet twice when using ``route-to`` leading to half the expected bandwidth
  • Status changed from Confirmed to Resolved
  • Assignee set to Kristof Provost
  • Target version set to 2.8.0
  • Plus Target Version set to 24.03

https://reviews.freebsd.org/D44365

A fix has been merged; I tested limiting with and without floating rules, as well as Captive Portal.

Actions #12

Updated by Jim Pingle about 1 month ago

  • Subject changed from packets are passed through dummynet twice when using ``route-to`` leading to half the expected bandwidth to Packets are passed through dummynet twice when using ``route-to`` leading to half the expected bandwidth
Actions

Also available in: Atom PDF