Bug #6200
closed
LACP with em driver does not work with cisco active lacp setup
0%
Description
LACP on the cisco switch does not get activated by LACP lagg on pfSense.
Using lagg with vlan on em nics. Hardware checksum offload is disabled in System->Advance->Networking
em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=42088<VLAN_MTU,VLAN_HWCSUM,WOL_MAGIC,VLAN_HWTSO>
ether 00:15:17:35:94:62
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
em1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=40088<VLAN_MTU,VLAN_HWCSUM,VLAN_HWTSO>
ether 00:15:17:35:94:62
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
bge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=80098<VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,LINKSTATE>
ether 00:50:8b:11:4c:3e
inet6 fe80::250:8bff:fe11:4c3e%bge0 prefixlen 64 scopeid 0x3
inet netmask 0xffffff80 broadcast
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
pflog0: flags=100<PROMISC> metric 0 mtu 33160
pfsync0: flags=0<> metric 0 mtu 1500
syncpeer: 224.0.0.240 maxupd: 128 defer: on
syncok: 1
enc0: flags=0<> metric 0 mtu 1536
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
inet 127.0.0.1 netmask 0xff000000
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x7
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
lagg0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=40088<VLAN_MTU,VLAN_HWCSUM,VLAN_HWTSO>
ether 00:15:17:35:94:62
inet 10.168.84.1 netmask 0xffffff00 broadcast 10.168.84.255
inet6 fe80::1:1%lagg0 prefixlen 64 scopeid 0x8
inet6 prefixlen 64
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
media: Ethernet autoselect
status: active
laggproto lacp lagghash l2,l3,l4
laggport: em0 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>
laggport: em1 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>
lagg0_vlan5: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
ether 00:15:17:35:94:62
inet6 fe80::215:17ff:fe35:9462%lagg0_vlan5 prefixlen 64 scopeid 0x9
inet 10.168.85.1 netmask 0xffffff00 broadcast 10.168.85.255
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
media: Ethernet autoselect
status: active
vlan: 5 vlanpcp: 0 parent interface: lagg0
lagg0_vlan3: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
ether 00:15:17:35:94:62
inet6 fe80::215:17ff:fe35:9462%lagg0_vlan3 prefixlen 64 scopeid 0xa
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
media: Ethernet autoselect
status: active
vlan: 3 vlanpcp: 0 parent interface: lagg0
lagg0_vlan6: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
ether 00:15:17:35:94:62
inet6 fe80::215:17ff:fe35:9462%lagg0_vlan6 prefixlen 64 scopeid 0xb
inet 10.168.86.1 netmask 0xffffff00 broadcast 10.168.86.255
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
media: Ethernet autoselect
status: active
vlan: 6 vlanpcp: 0 parent interface: lagg0
lagg0_vlan10: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
ether 00:15:17:35:94:62
inet6 fe80::215:17ff:fe35:9462%lagg0_vlan10 prefixlen 64 scopeid 0xc
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
media: Ethernet autoselect
status: active
vlan: 10 vlanpcp: 0 parent interface: lagg0
lagg0_vlan50: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
ether 00:15:17:35:94:62
inet6 fe80::215:17ff:fe35:9462%lagg0_vlan50 prefixlen 64 scopeid 0xd
inet 10.168.130.1 netmask 0xffffff00 broadcast 10.168.130.255
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
media: Ethernet autoselect
status: active
vlan: 50 vlanpcp: 0 parent interface: lagg0
ovpns1: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> metric 0 mtu 1500
options=80000<LINKSTATE>
inet6 fe80::250:8bff:fe11:4c3e%ovpns1 prefixlen 64 scopeid 0xf
inet 10.168.88.1 --> 10.168.88.2 netmask 0xffffffff
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
Opened by PID 20185
wan_stf: flags=4041<UP,RUNNING,LINK2> metric 0 mtu 1280
inet6 prefixlen 32
nd6 options=1<PERFORMNUD>
v4net
cisco lacp status:
LACP port Admin Oper Port Port
Port Flags State Priority Key Key Number State
Gi0/23 SA indep 32768 0x1 0x1 0x118 0x7D
Gi0/24 SA indep 32768 0x1 0x1 0x119 0x7D
Files
Updated by Chris Buechler over 8 years ago
- Status changed from New to Feedback
What version?
Add a tunable in System>Advanced, System Tunables, net.link.lagg.lacp.debug value 1. Post the system logs you get after doing so.
Updated by David Wood over 8 years ago
The original poster has started the forum thread https://forum.pfsense.org/index.php?topic=110326.msg0
The thread states that this issue is with 2.3.
Updated by Enzo Matrix over 8 years ago
Yes, I am currently on 2.3. But this feature has never worked with any prior version I used, i.e. 2.x.x
Updated by Chris Buechler over 8 years ago
- Category set to Operating System
- Affected Version set to All
Seems sane. What about "debug lacp all" from the switch?
Updated by Enzo Matrix over 8 years ago
Apr 20 11:58:49.512: %SW_MATM-4-MACFLAP_NOTIF: Host 0015.1735.9462 in vlan 1 is flapping between port Gi0/24 and port Gi0/23
Apr 20 11:58:53.161: %SW_MATM-4-MACFLAP_NOTIF: Host 0015.1735.9462 in vlan 50 is flapping between port Gi0/24 and port Gi0/23
Apr 20 11:58:55.334: LACP: lacp_p(Gi0/23) timer stopped
Apr 20 11:58:55.334: LACP: lacp_p(Gi0/23) expired
Apr 20 11:58:55.334: lacp_ptx Gi0/23 - ptx: during state SLOW_PERIODIC, got event 3(pt_expired)
Apr 20 11:58:55.334: @ lacp_ptx Gi0/23 - ptx: SLOW_PERIODIC -> PERIODIC_TX
Apr 20 11:58:55.334: LACP: Gi0/23 lacp_action_ptx_slow_periodic_exit entered
Apr 20 11:58:55.334: LACP: lacp_p(Gi0/23) timer stopped
Apr 20 11:58:55.334: LACP: Gi0/23 lacp_action_ptx_periodic_tx entered
Apr 20 11:58:55.334: LACP: timer lacp_t(Gi0/23) started with interval 1000.
Apr 20 11:58:55.334: LACP: lacp_insert_partner_cd_inhibitor: didn't change sync flag.
Apr 20 11:58:55.334: LACP: lacp_send_lacpdu: (Gi0/23) About to send the 110 LACPDU
Apr 20 11:58:55.334: LACP :lacp_bugpak: Send LACP-PDU packet via Gi0/23
Apr 20 11:58:55.334: LACP : packet size: 124
Apr 20 11:58:55.334: LACP: pdu: subtype: 1, version: 1
Apr 20 11:58:55.334: LACP: Act: tlv:1, tlv-len:20, key:0x1, p-pri:0x8000, p:0x118, p-state:0x7D,
s-pri:0x8000, s-mac:001e.bd01.f200
Apr 20 11:58:55.334: LACP: Part: tlv:2, tlv-len:20, key:0x0, p-pri:0x0, p:0x0, p-state:0x0,
s-pri:0x0, s-mac:0000.0000.0000
Apr 20 11:58:55.334: LACP: col-tlv:3, col-tlv-len:16, col-max-d:0x8000
lacp_rx Gi0/19 - rx: during state CURRENT, got event 5(recv_lacpdu)
Apr 20 11:59:11.566: @ lacp_rx Gi0/19 - rx: CURRENT -> CURRENT
Apr 20 11:59:11.566: LACP: Gi0/19 lacp_action_rx_current entered
Apr 20 11:59:11.566: LACP: recordPDU Gi0/19 LACP PDU Rcvd. Partners oper state is hex 3D
Apr 20 11:59:11.566: LACP: recordPDU Gi0/19 Partner in sync and aggregating
Apr 20 11:59:11.566: LACP: Gi0/19 Partners oper state is hex 3D
Apr 20 11:59:11.566: LACP: timer lacp_c_l(Gi0/19) started with interval 90000.
Apr 20 11:59:11.566: LACP: Gi0/19 LAG_PARTNER_UP.
Apr 20 11:59:11.566: LACP: Gi0/19 LAG unchanged
Apr 20 11:59:11.566: lacp_mux Gi0/19 - mux: during state COLLECTING_DISTRIBUTING, got event 5(in_sync) (ignored)h interval 1000.
Apr 20 11:59:08.076: LACP: lacp_insert_partner_cd_inhibitor: didn't change sync flag.
Apr 20 11:59:08.076: LACP: lacp_send_lacpdu: (Gi0/19) About to send the 110 LACPDU
Apr 20 11:59:08.076: LACP :lacp_bugpak: Send LACP-PDU packet via Gi0/19
Apr 20 11:59:08.076: LACP : packet size: 124
Apr 20 11:59:08.076: LACP: pdu: subtype: 1, version: 1
Apr 20 11:59:08.076: LACP: Act: tlv:1, tlv-len:20, key:0x2, p-pri:0x8000, p:0x114, p-state:0x3C,
s-pri:0x8000, s-mac:001e.bd01.f200
Apr 20 11:59:08.076: LACP: Part: tlv:2, tlv-len:20, key:0x9, p-pri:0xFF, p:0x2, p-state:0x3D,
s-pri:0xFFFF, s-mac:1c6f.65c3.069d
Apr 20 11:59:08.076: LACP: col-tlv:3, col-tlv-len:16, col-max-d:0x8000
Apr 20 11:59:08.076: LACP: term-tlv:0 termr-tlv-len:0
Apr 20 11:59:08.076: LACP: lacp_write: LACP 124 bytes out Gi0/19
Apr 20 11:59:08.076: lacp_ptx Gi0/19 - ptx: during state PERIODIC_TX, got event 2(long_timeout)
Apr 20 11:59:08.076: @ lacp_ptx Gi0/19 - ptx: PERIODIC_TX -> SLOW_PERIODIC
Apr 20 11:59:08.076: LACP: Gi0/19 lacp_action_ptx_slow_periodic entered
Apr 20 11:59:08.076: LACP: timer lacp_p_s(Gi0/19) started with interval 30000.
Apr 20 11:59:08.831: LACP: lacp_t(Gi0/24) timer stopped
Apr 20 11:59:08.831: LACP: lacp_t(Gi0/24) expired
Updated by Enzo Matrix over 8 years ago
The problem is on Gi0/23 and Gi0/24. These are the teamed ports connected to pfSense.
Updated by
Updated by Jim Pingle about 8 years ago
- Assignee deleted (
Jim Pingle) - Priority changed from High to Normal
I don't have a Cisco switch to test against, and the only piece of hardware I have left with em0 that works is 32-bit only.
I can't replicate any problems with em0 on that box running 2.3.2 with active LACP against my TP-Link switch, it works fine there. All ports come up and show active.
This must be specific to Cisco, assuming it's still a problem. If someone who can replicate the issue could try it on 2.4 that would be a much better test. Or better yet, try the same hardware on stock FreeBSD. If it's a problem on stock FreeBSD, report it upstream. Make sure you're on a current switch firmware as well before testing there.
Lowering priority as it appears there is nothing we can do to help or hurt here, it looks like it is between Cisco and the OS/NIC driver, possibly even this specific combination of switch and NIC chipset.
Updated by Jim Pingle about 7 years ago
- Status changed from Feedback to Not a Bug
This is working fine on current versions and no additional feedback. Closing.