Bug #15363
closed
Reply traffic on a secondary WAN may be dropped when passed through dummynet
100%
Description
When a dummynet pipe with a delay is applied to traffic on a secondary WAN, reply traffic is dropped. It seems that the fix in #15220 does not take effect in this scenario.
Test setup:vmx1 is WAN1, vmx2 is WAN2
# match rule -- pfctl -vvsr @296 match in on vmx2 inet all label "USER_RULE: QoS queue default (outside) IPv4" label "id:1686509600" ridentifier 1686509600 dnqueue(12, 9) ! tagged blocklist [ Evaluations: 151142 Packets: 284 Bytes: 78078 States: 0 ] [ Inserted: uid 0 pid 0 State Creations: 0 ] [ Last Active Time: N/A ] # pass rule -- pfctl -vvsr @799 pass in quick on vmx2 reply-to (vmx2 192.168.1.254) inet proto udp from any to 127.0.0.1 port = rsf-1 keep state (if-bound) label "USER_RULE: OpenVPN" label "id:1679170153" ridentifier 1679170153 ! tagged blocklist [ Evaluations: 438 Packets: 301 Bytes: 106093 States: 0 ] [ Inserted: uid 0 pid 0 State Creations: 2 ] [ Last Active Time: Tue Mar 26 12:24:48 2024 ]
The following works: limiter queue without a delay on the pipe:
# pipe without delay -- dnctl pipe show 00004: 80.000 Mbit/s 0 ms burst 0 q131076 50 sl. 0 flows (1 buckets) sched 65540 weight 0 lmax 0 pri 0 droptail sched 65540 type FIFO flags 0x0 0 buckets 0 active # state info -- pfctl -vvss vmx2 udp 127.0.0.1:1195 (192.168.1.253:1195) <- 172.58.109.152:61712 MULTIPLE:MULTIPLE age 00:00:11, expires in 00:00:51, 10:8 pkts, 3632:3280 bytes, rule 799 id: dd6b0a6600000000 creatorid: af6c8b55 reply-to: 192.168.1.254@vmx2 origif: vmx1
The following does not work: limiter queue with a 1ms delay on the pipe:
# pipe with 1ms delay -- dnctl pipe show 00004: 80.000 Mbit/s 1 ms burst 0 q131076 50 sl. 0 flows (1 buckets) sched 65540 weight 0 lmax 0 pri 0 droptail sched 65540 type FIFO flags 0x0 0 buckets 0 active # state info -- pfctl -vvss all udp 127.0.0.1:1195 (192.168.1.253:1195) <- 172.58.109.152:64462 NO_TRAFFIC:SINGLE age 00:00:40, expires in 00:00:20, 5:0 pkts, 410:0 bytes, rule 799 id: 7fe5096600000000 creatorid: af6c8b55 reply-to: 192.168.1.254@vmx2 origif: vmx2
Related issues
Updated by Marcos M 9 months ago
- Related to Todo #15220: Handle ``route-to`` and ``reply-to`` states when using the ``if-bound`` state policy added
Updated by Kristof Provost 9 months ago
I'll try to reproduce this.
First thoughts:
The state being created as floating (i.e. 'all') is expected for reply-to. The fix in https://cgit.freebsd.org/src/commit?id=6460322a0a512f4e2c263bee54fc8bf46091f4cd means we initially create reply-to states as floating, and later bind them to the appropriate interface, when we get the reply and can figure out what interface it ought to bind to.
That it remains as 'all' would suggest that there's no reply or something else goes wrong handling the reply.
Secondly, the distinction between a limiter without delay and one with delay is probably because the limiter did not hold on to the packet for out-of-flow processing. Dummynet can, depending on circumstances, either pass through the packet and allow the normal flow to continue, or adopt the packet and later re-inject it. This distinction probably means that whatever the issue is, it happens in the re-injection path. It will almost certainly also happen if the limiter actually applies (i.e. if there's more traffic than the limiter would allow).
Updated by Kristof Provost 9 months ago
Fixed in https://cgit.freebsd.org/src/commit/?id=a983cea4e9a8dcd52cfd6a3141d7aa03306b057b (and cherry-picked to plus-devel-main).
Updated by Kristof Provost 9 months ago
- Status changed from Confirmed to Feedback