pfSense

It looks like this issue still happens in FreeBSD Head. Though unlike in pfSense (FreeBSD 12) we can see outbound traffic in packet captures. Replies still come back unexpectedlu double tagged though:

17:29:30.370205 90:ec:77:1f:8a:5f > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 46: vlan 1001, p 0, ethertype ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.101.0.12 tell 10.101.0.1, length 28
17:29:30.370787 00:51:82:11:22:02 > 90:ec:77:1f:8a:5f, ethertype 802.1Q (0x8100), length 64: vlan 0, p 0, ethertype 802.1Q, vlan 1001, p 0, ethertype ARP, Ethernet (len 6), IPv4 (len 4), Reply 10.101.0.12 is-at 00:51:82:11:22:02, length 42

Hence packets are still dropped and the connection fails.

Tested: FreeBSD-14.0-CURRENT-amd64-20220729-467d3e2e8aa-257025-memstick.img

I've been able to reproduce this (on pfsense/main).

That required the following:

ifconfig vlan create vlandev ix3 vlan 42
ifconfig vlan0 192.168.42.1/24 up
ifconfig ix3 -vlanhwtag

The traffic sent out through the interface it fine, but received traffic is incorrectly double-tagged (once with vlan 0, then with vlan 42. i.e. the outer tag is 0, the inner is 42).

Reverting the listed patch (9c762cc125c0c2dae9fbf49cc526bb97c14b54a4) fixes the problem.

Interestingly the problem does not occur if the vlanhwtag feature is disabled before the vlan is created. I believe that to be an important clue.

I proposed a patch in https://reviews.freebsd.org/D36139
It works for me, but I'd like the Intel people (and driver maintainers) to take a look before I commit it.

Steve Wheeler wrote:

VLAN tagged traffic fails on an ix NIC if hardware vlan tagging is disabled.
For example:
[...]

VLAN hardware tagging is enabled by default so this is not easy to hit.

It produces some unexpected behaviour. In a packet capture on the parent interface there is no outbound VLAN traffic show at all.
Inbound VLAN traffic appears as double tagged with VLAN0 as the outer tag:
[...]

VLAN0 is expected to be dropped.

This behaviour appears to have been introduced by this commit:
https://github.com/pfsense/FreeBSD-src/commit/9c762cc125c0c2dae9fbf49cc526bb97c14b54a4
All snapshots after 20220314-1916 exhibiting it.

Tested on a 4100 on 22.05 and in 2.7. The user who hit this initially is also using a C3K SoC device with the same on-board NICs.

See: https://forum.netgate.com/topic/173149/pfsense-22-05-breaks-vlans-restoring-pfsense-22-01-fixes-the-issue

Hi, I am having a similar problem with ixl interfaces, In my case, I can't reach any device in the VLAN only the router (pfsense):

ixl1: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        description: LAN
        options=a100b9<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWFILTER,RXCSUM_IPV6>
        ether 64:9d:99:b1:80:03
        inet6 fe80::669d:99ff:feb1:8003%ixl1 prefixlen 64 scopeid 0x2
        inet6 2a0e:97c0:620:affe::1 prefixlen 64
        inet 192.168.0.1 netmask 0xffffff00 broadcast 192.168.0.255
        media: Ethernet autoselect (10Gbase-Twinax <full-duplex>)
        status: active
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
        plugged: SFP/SFP+/SFP28 1X Copper Passive (Copper pigtail)

ixl1.20: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        description: bhyve
        options=200001<RXCSUM,RXCSUM_IPV6>
        ether 64:9d:99:b1:80:03
        inet6 fe80::669d:99ff:feb1:8003%ixl1.20 prefixlen 64 scopeid 0x11
        inet 10.0.0.1 netmask 0xffffff00 broadcast 10.0.0.255
        groups: vlan
        vlan: 20 vlanpcp: 0 parent interface: ixl1
        media: Ethernet autoselect (10Gbase-Twinax <full-duplex>)
        status: active
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>