Actions
Bug #15606
closed
Data transfer problems when using interface-bound states with automatic floating states for IPsec rules
Start date:
Due date:
% Done:
100%
Estimated time:
Plus Target Version:
24.11
Release Notes:
Force Exclusion
Affected Version:
2.8.0
Affected Architecture:
Description
Version: 24.03-RELEASE (amd64)
Platform: PVE/KVM
Test environment:
linux <--> pfSense <-- (IPsec VTI) --> pfSense <--> linux
Baseline test of iperf3 and sftp (1GB file) using floating states . (linux --> linux)
Accepted connection from 10.11.11.12, port 55973 [ 5] local 10.10.10.11 port 5201 connected to 10.11.11.12 port 53279 [ ID] Interval Transfer Bitrate [ 5] 0.00-1.00 sec 81.6 MBytes 684 Mbits/sec [ 5] 1.00-2.00 sec 72.6 MBytes 609 Mbits/sec [ 5] 2.00-3.00 sec 61.9 MBytes 519 Mbits/sec [ 5] 3.00-4.00 sec 63.9 MBytes 536 Mbits/sec [ 5] 4.00-5.00 sec 72.2 MBytes 605 Mbits/sec [ 5] 5.00-6.00 sec 70.1 MBytes 588 Mbits/sec [ 5] 6.00-7.00 sec 61.8 MBytes 518 Mbits/sec [ 5] 7.00-8.00 sec 66.9 MBytes 561 Mbits/sec [ 5] 8.00-9.00 sec 82.9 MBytes 695 Mbits/sec [ 5] 9.00-10.00 sec 62.3 MBytes 522 Mbits/sec [ 5] 10.00-10.00 sec 171 KBytes 494 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate [ 5] 0.00-10.00 sec 696 MBytes 584 Mbits/sec receiver
sftp> put tstfile Uploading tstfile to /home/ccoonrad/tstfile tstfile 100% 1024MB 64.9MB/s 00:15
Test switching state policy to interface with patch #15430 applied.
Accepted connection from 10.11.11.12, port 58451 [ 5] local 10.10.10.11 port 5201 connected to 10.11.11.12 port 54957 [ ID] Interval Transfer Bitrate [ 5] 0.00-1.00 sec 64.5 KBytes 528 Kbits/sec [ 5] 1.00-2.00 sec 0.00 Bytes 0.00 bits/sec [ 5] 2.00-3.00 sec 0.00 Bytes 0.00 bits/sec [ 5] 3.00-4.00 sec 0.00 Bytes 0.00 bits/sec [ 5] 4.00-5.00 sec 0.00 Bytes 0.00 bits/sec [ 5] 5.00-6.00 sec 0.00 Bytes 0.00 bits/sec [ 5] 6.00-7.00 sec 0.00 Bytes 0.00 bits/sec [ 5] 7.00-8.00 sec 0.00 Bytes 0.00 bits/sec [ 5] 8.00-9.00 sec 0.00 Bytes 0.00 bits/sec [ 5] 9.00-10.00 sec 0.00 Bytes 0.00 bits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate [ 5] 0.00-10.00 sec 64.5 KBytes 52.8 Kbits/sec receiver
sftp transfer starts in a stalled state (eventually something times out, and the transfer completes.)
sftp> put tstfile Uploading tstfile to /home/ccoonrad/tstfile tstfile 0% 0 0.0KB/s - stalled -
The pcaps for the iperf3 and sftp sessions both show the receiver sending a bunch of ACKs, followed by a bunch of TCP retransmits from the sender.
sender: 10.11.11.12 receiver: 10.10.10.11
Files
Updated by Jim Pingle 5 months ago
- Subject changed from Data transfer problems with patch #15430 (Automatically use floating states for IPsec rules) to Data transfer problems when using interface-bound states with automatic floating states for IPsec rules
- Assignee set to Marcos M
Updated by Marcos M 5 months ago
- Project changed from pfSense Plus to pfSense
- Category changed from IPsec to IPsec
- Status changed from New to Ready To Test
- Target version set to 2.8.0
- Plus Target Version set to 24.08
This seems to happen because of the bogus state that's created initially on the VTI, e.g.:
ipsec1 icmp 10.255.5.1:61473 -> 10.255.5.2:61473 0:0 age 00:00:01, expires in 00:00:19, 1:0 pkts, 84:0 bytes, rule 102, allow-opts id: 4fab476600000000 creatorid: 3152b202 all icmp 10.255.5.1:61473 -> 10.255.5.2:61473 0:0 age 00:00:01, expires in 00:00:09, 1:1 pkts, 84:84 bytes, rule 114 id: 50ab476600000000 creatorid: 3152b202 origif: enc0
Once the state expires (the one on ipsec1 for the example above), traffic matches the all state instead and succeeds.
This results in the initial stall behavior for the sftp transfer. The iperf test similarly stalls and then succeeds if left running for longer than the state expiration.
https://gitlab.netgate.com/pfSense/pfSense/-/merge_requests/1162
Somewhat oddly, testing shows this is not always an issue. For example, downloading/uploading a file via HTTPS works.
Updated by Craig Coonrad 5 months ago
Tested Marcos' patch successfully <thumbs up>
[ ID] Interval Transfer Bitrate Retr [ 5] 0.00-60.00 sec 3.67 GBytes 525 Mbits/sec 814 sender [ 5] 0.00-60.01 sec 3.67 GBytes 525 Mbits/sec receiver
sftp> put tstfile tstfile 100% 1024MB 66.4MB/s 00:15
Updated by Marcos M 5 months ago
- Status changed from Ready To Test to Feedback
- % Done changed from 0 to 100
Applied in changeset 3b3be7348bdf0f75d474a6aec938d8143e90c8bf.
Updated by Jim Pingle about 1 month ago
- Plus Target Version changed from 24.08 to 24.11
Actions