Project

General

Profile

Actions
Copy link

Todo #14207

closed

Rate limiting on Chelsio T4/5 NICs

Added by Bruce Talbot about 1 year ago. Updated 10 months ago.

Status:
Resolved
Priority:
Normal
Assignee:
Jim Pingle
Category:
Hardware
Target version:
-
Start date:
Due date:
% Done:

0%

Estimated time:

Description

Chelsio T520-CR and T420-CR are unable to route speeds over 470mbps when updated to 23.01 code. Goes to full 1gb speed after reverting back to 22.05. WAN is on Intel i210-AT chip, LAN is on 10gb Chelsio on a DAC cable going into 10gb switch. Other devices on network are also running 10Gb.
Hardware:
Supermicro SYS-5019S-M (https://www.supermicro.com/en/products/system/1U/5019/SYS-5019S-M.cfm)
Intel(R) Xeon(R) CPU E3-1220 v5
Chelsio T520-CR

1gb wan on copper
10gb LAN on DAC (also running vlans router-on-a-stick on this interface)

Initial postings were on Netgate forums (https://forum.netgate.com/topic/177885/upgrade-to-23-01-wan-speed-halved/26?_=1679281728569)

I did test with putting in a Chelsio t420 card with similar results. Additionally throwing a 1gb sfp copper module in the Chelsio T520 and changing the WAN to it and was still having the same results.

10g to 10g test run at full speed. Just 10gb to 1gb are impacted, even when done from 10gb client to 1gb client on LAN (not going out the WAN).

This is a duplicate entry as the first bug report I put in incorrectly.

Actions
Copy link
 #1

Updated by Steve Wheeler 12 months ago

  • Status changed from New to Confirmed

This appears to affect current 2.7 and 23.05 snapshots.

The -SO hardware variants seem unaffected

Actions
Copy link
 #2

Updated by Steve Wheeler 11 months ago

  • Target version set to 2.7.0
  • Plus Target Version set to 23.09

Anyone hitting this should try the loader variable: 

hw.cxgbe.niccaps_allowed="1"

Actions
Copy link
 #3

Updated by Bruce Talbot 11 months ago

Tested the fix posted above which fixed this issue.

Actions
Copy link
 #4

Updated by Mike Schwier 11 months ago

Confirmed this fixed the issue with a Chelsio T520-CR on 23.05

Actions
Copy link
 #5

Updated by Mike Schwier 11 months ago

Actions
Copy link
 #6

Updated by Jim Pingle 11 months ago

Mike Schwier wrote in #note-5:

This should be added to the tuning documentation at https://docs.netgate.com/pfsense/en/latest/hardware/tune.html#chelsio-cxgbe-4-cards

I added a note about that setting to the docs: https://gitlab.netgate.com/docs/pfSense-docs/-/commit/4a3f10e4d0c8f4abe0c6922b3c65331bcaefa824

It will show up in ~10 minutes once the docs build and deploy.

Actions
Copy link
 #7

Updated by Marcos M 10 months ago

  • Tracker changed from Bug to Todo
  • Project changed from pfSense to pfSense Docs
  • Subject changed from 470Mbps max speed on Chelsio T4/5 cards to Rate limiting on Chelsio T4/5 NICs
  • Category changed from Hardware / Drivers to Hardware
  • Status changed from Confirmed to Resolved
  • Assignee set to Jim Pingle
  • Target version deleted (2.7.0)
  • Plus Target Version deleted (23.09)
  • Release Notes deleted (Default)
  • Affected Version deleted (2.7.0)
  • Affected Architecture deleted (amd64)

Changing issue scope as this does not seem to be a bug with pfSense software.

Actions
Copy link

Also available in: Atom PDF