pfSense

Hi again.

I restored config on 2.4.4-p3 and this are working just fine there.

I believe this on is related to https://redmine.pfsense.org/issues/8954#change-41002 .

Thanks!

Hmmm OK, I have Hyper-V, 2.5.0 and pppoe.

But weird is, that on when applied on IN direction on LAN it works ok.

I experienced the same behavior as Greg M when updating from 2.4.4-p3 to 2.5.0. This was on a bare-metal install.

Hi.

Any update on this one?

Thanks!

Nothing yet, but since we are rebasing on FreeBSD 12.1 soon, it will need to wait until after that happens.

Experiencing same behavior as reported by Greg M on my physical install of pfsense 2.5 dev. Traffic just stops.

Followed the instructions from 2.4.4 here (from a pfsense Short Topic vid, first topic): https://www.youtube.com/watch?v=o8nL81DzTlU&t=380 No dice.

Meantime, I disabled the floating rule for now, and I try again every few weeks...no change in behavior since I first tried back at the start of Jan 2020.

Any news? Thanks, in advance. Love your software otherwise! Gorgeous, powerful, versatile!

Hi.

Same behavior on Proxmox/KVM - pfSense 2.5.0.a.20200518.1031 with vtnet driver.
Any update on this?

Thanks!

not working for me either
2.5.0.a.20200522.0732
I need to disable the floating rule to make internet work again

I'm having the same issue, running on a VK-T40E:

2.5.0.a.20200603.1253

If I enable the floating rule, I lose all internet/WAN traffic.

Out of interest, for those running on 2.4.x, can you see the queue in any of the pfSense monitoring/status/log tools?

Hi guys, just to confirm it looks like I have the same problem.

pfSense running in a Proxmox VM, I did gui update from 2.4.x to 2.5 devel.

After an hour I was informed server's behind firewall are dead, and noticed dummynet was making all traffic just timeout as if there was a block rule in place, the limiter was configured as in the pfSense video guide.

I am using vtnet driver.

I can also confirm it works fine on LAN, and since the setup uses NAT, it means I can use this as a workaround, I put myself forward as a volunteer if any patches get made and need testing, I can test on this server.

Still broken for me on the latest build, I tested just now.

Can you give me more details ? show me your rules and results ?

Here are the limiters and firewall floating rule I used. When the firewall rule is enabled, no traffic gets through the WAN, the same setup worked fine in 2.4.5-P1. I have no other user generated rulesets other than default from install time. No pfBlockerng or any other packages. I am testing bare metal on an AMD Epyc 3251 SuperMicro build w/Intel X710-T2L ethernet adapter.

UPDATE: The upload limiter seems to work, I went back to the firewall rule and selected none for the "Out Pipe". So basically it's the downlink limiter that's not functional in 2.5.0.

Limiters:

Download Limiter:
Bandwidth: 1000 Mbps (I get 1150-1200 Mbps from ISP) (later tried 900 Mbps)
Queue Management Algorithm: CoDel
Scheduler: FQ_Codel
Queue Length: 1000 also tried leaving blank
ECN: Enabled
—Download Queue:
Queue Management Algorithm: CoDel
ECN: Enabled

Upload Limiter:
Bandwidth: 36 Mbps
Queue Management Algorithm: CoDel
Scheduler: FQ_Codel
Queue Length: 1000 also tried leaving blank
ECN: Enabled
—Upload Queue:
Queue Management Algorithm: CoDel
ECN: Enabled

Firewall Floating Rule:
Action: Pass
Interface: WAN
Direction: Out
Address Family: IPv4
Protocol: Any
Advanced:
Gateway: WAN_DHCP - 192.168.x.x
In/Out Pipe: Upload Queue (In) / Download Queue (Out)

Abhinav Tella wrote:

Here are the limiters and firewall floating rule I used. When the firewall rule is enabled, no traffic gets through the WAN, the same setup worked fine in 2.4.5-P1. I have no other user generated rulesets other than default from install time. No pfBlockerng or any other packages. I am testing bare metal on an AMD Epyc 3251 SuperMicro build w/Intel X710-T2L ethernet adapter.

UPDATE: The upload limiter seems to work, I went back to the firewall rule and selected none for the "Out Pipe". So basically it's the downlink limiter that's not functional in 2.5.0.

Limiters:

Download Limiter:
Bandwidth: 1000 Mbps (I get 1150-1200 Mbps from ISP) (later tried 900 Mbps)
Queue Management Algorithm: CoDel
Scheduler: FQ_Codel
Queue Length: 1000 also tried leaving blank
ECN: Enabled
—Download Queue:
Queue Management Algorithm: CoDel
ECN: Enabled

Upload Limiter:
Bandwidth: 36 Mbps
Queue Management Algorithm: CoDel
Scheduler: FQ_Codel
Queue Length: 1000 also tried leaving blank
ECN: Enabled
—Upload Queue:
Queue Management Algorithm: CoDel
ECN: Enabled

Firewall Floating Rule:
Action: Pass
Interface: WAN
Direction: Out
Address Family: IPv4
Protocol: Any
Advanced:
Gateway: WAN_DHCP - 192.168.x.x
In/Out Pipe: Upload Queue (In) / Download Queue (Out)

I am also having the same issue running the identical configuration as Abhinav Tella on 2.5.0-DEVELOPMENT (amd64) (built on Thu Sep 10 01:03:22 EDT 2020). The only difference is that I have it split between two floating rules; one for IPv4 and the other for IPv6.

Is there any update on this?

Tested fq-codel out on the latest snapshot and found out that if i apply an outbound WAN pass rule to ipv6 it does apply and works just fine for IPv6 traffic. But at the same time IPv4 traffic gets instant blocked even though there isn't a rule that apply for it

Hi. I am also able to reproduce this. It works fine on 2.4.5, but on 2.5.0, the minute the floating rule is enable, I lose access.

Peter Grehan wrote:

I'm able to reproduce this. As mentioned in earlier comments, the issue only shows when the inbound queue is enabled. Will be doing some more tracing of ipfw/dummynet to see where packets are being dropped.

Luiz Souza wrote:

All the fixes from 2.4.5 are now merged.

Initial tests looks good.

I can confirm this is resolved for me on 2.5.0.r.20210210.0933.

Thank you and congrats on the release!

I can confirm working too.

Looks good here as well. Not only can I pass traffic with limiters on, I am back to an A on the bufferbloat test thanks to CoDel.

I'm not sure if this is related, my understanding of this is limited. After creating/enabling CODEL traffic limiters again after updating pfSense to 2.5.0 RC and applying floating rules for IPv4 and IPv6, I receive the following error message:

There were error(s) loading the rules: /tmp/rules.debug:326: no routing address with matching address family found. - The line in question reads [326]: pass out quick on { igb0 } $GWWAN_DHCP6 inet6 from any to any tracker 1609523960 keep state dnqueue( 2,1) label "USER_RULE: CoDel Limiters"
@ 2021-02-11 07:39:56

If I disable the IPv6 floating rule for the limiter I do not receive the error message, but upon re-enabling the rule the error comes back.

That doesn't appear to be related to this specific issue, it looks like a problem with your rule / state of your system.

Jim Pingle wrote:

That doesn't appear to be related to this specific issue, it looks like a problem with your rule / state of your system.

Any guidance on where I could look for assistance so I don't clog up this bug report?

Make a post on the forum and discuss it there, that's the best way to diagnose your issue.

I updated to the latest stable (new RC 2.5)

Sadly I still have the same problem, I am still checking stuff to make sure I haven't broken it from previous fiddling to try and get it working when it initially broke but early signs are not good.

Its basically floating match rule on WAN interface, quick match, assign the limiter, and all traffic hitting the rule is going into a black hole.

Ok to summarise.

It works on outbound WAN matching (this was broken before the patch).
It works on inbound and outbound LAN matching.
It goes into a black hole on inbound WAN matching.

If I keep the match rule but remove the dummynet queue (so it just matches but does nothing elsE), the traffic is processed normally, so it is a dummynet issue on inbound WAN traffic.

Chris Collins wrote:

Ok to summarise.

It works on outbound WAN matching (this was broken before the patch).
It works on inbound and outbound LAN matching.
It goes into a black hole on inbound WAN matching.

If I keep the match rule but remove the dummynet queue (so it just matches but does nothing elsE), the traffic is processed normally, so it is a dummynet issue on inbound WAN traffic.

Is that with both IPv4 and IPv6 rules?

I only have it configured with ipv4.

Chris Collins wrote:

It goes into a black hole on inbound WAN matching.

If I keep the match rule but remove the dummynet queue (so it just matches but does nothing elsE), the traffic is processed normally, so it is a dummynet issue on inbound WAN traffic.

Was that same configuration known to work on 2.4.5-p1? That doesn't sound like something we've supported with limiters (there are some quirks with match+dummynet behavior) -- Nobody else is reporting issues, including people with limiters on multiple WANs, so it may be you have stumbled onto a case that shouldn't work which wouldn't be a valid test for this bug.

It did work yes, the reason for the configuration is, the firewall is in front of a webserver, and I want people who download the files to not have an advantage if they use something like a download manager to multithread, so I shape per ip, and I only want it to shape on webserver traffic, so I match the connections inbound.

If you want, I can probably setup another pfsense VM fairly quickly on 2.4.5 just to confirm this for this issue.

I am able to do the shaping on LAN though so it isnt the end of the world if its not working, but you may possibly get other reports if anyone else uses it the same way as I was.

At this point I'd say open a new and more specific bug report for that once you have all the info collected and re-tested, since this issue was for limiters being completely broken for everybody and that is a much more niche/focused case which seems unrelated. Something else in the base OS changing might have affected that differently.

Yeah I can do that at a later date. I will keep it out of this report now.

I believe I have the same issue, I just upgraded from 2.4.5 to 2.5.0 and upload queues are empty.
I also use multi-WAN configuration.

An example of floating rule that matched HTTPS traffic:
Action: match
Interface: WAN1
Direction: any
Family: IPv4+IPv6
Protocol: TCP
Source: any
Destination: any, port 443
Ackqueue: qACK1
Queue: qNormal1

Another one was identical with these differences:
Interface: WAN2
Ackqueue: qACK2
Queue: qNormal2

Queues have this structure:

Interface WAN1
-qUploadV1    
--qACK1    
--qICMP1    
--qRealtime1    
--qHigh1    
--qNormal1    
--qLow1    

Interface WAN2
-qUploadV2    
--qACK2    
--qICMP2    
--qRealtime2    
--qHigh2    
--qNormal2    
--qLow2    

Interface LAN
-qLink    
--qDownloadV1    
---qACK1    
---qICMP1    
---qRealtime1    
---qHigh1    
---qNormal1    
---qLow1    
--qDownloadV2    
---qACK2    
---qICMP2    
---qRealtime2    
---qHigh2    
---qNormal2    
---qLow2

On version 2.4.5 and before, download and upload queues were filled with this traffic. On 2.5.0 the upload queues are empty, effectively not shaping upload at all causing increased latency over DSL lines.
Any advice please?