pfSense

After extensive research I found out why a Netgate XG1537 is unable to saturate a fiber WAN link. It turns out this is due to the old and much less performant TCP Stack of PfSense+ 24.03 compared to freeBSD 15. I purchased a Netgate XG1537 explicitly for it's hardware performance and software support of HAProxy and OpenVPN. It it used as edge router/firewall and was supposed to provide a reverse proxy service for two http(s) servers behind it.

The issue with the TCP stack is that it does not handle TCP packet reordering very well. So it incorrectly detects 2-3% packet los, fast retransmits a lot of packets and reduces throughput. This limits TCP upload to about 2-8MBit/s on a 400+MBit/s uplink (see screenshot). The solution to this problem is the Recent Acknowledgement algorithm for TCP (RACK TCP), which is the default in current Linux kernels and can also be activated in stock freeBSD 14+. Unfortunately the PfSense kernel does not come with alternative TCP stacks but only with the default one.

https://freebsdfoundation.org/our-work/journal/browser-based-edition/networking-10th-anniversary/rack-and-alternate-tcp-stacks-for-freebsd/

I ask you to include the RACK TCP Stack from stock freeBSD in the next PfSense release. Without it, TCP is pretty much unusable for any production service. Sure, PfSense is primarily a firewall, but your official documentation indicates support for HAProxy, OpenVPN and other services which (may) use TCP. These are all obsolete and cannot be used reliably. Without the support for high performance TCP (honestly I can't believe that's actually an issue in 2024), users will have to switch to another Operating System, which would be a shame. I for one would much prefer to stay with your software as it is pretty solid and reliable otherwise. But in my case, RHEL (and probably any other current Linux) gets solid 400-500MBit/s upload with ZERO retransmissions after establishing the connection out of the WAN (see screenshot).

With faster network connections the problem of TCP packet reordering will only occur more frequently - making the need for RACK TCP more urgent.