Bug #7801
closedUDP fragments received over IPsec tunnel are not properly reassembled and forwarded
100%
Description
I used this to generate the large UDP datagrams on Host J1: nping —udp —source-port 5060 —dest-port 5060 —data-length 2000 172.31.200.100 or nping —udp —source-port 5060 —dest-port 5060 —data-length 2000 172.31.202.100 With pf enabled on both H and J Into J LAN 01:12:22.571417 IP 172.31.201.100.5060 > 172.31.200.100.5060: UDP, length 2000 01:12:22.571425 IP 172.31.201.100 > 172.31.200.100: ip-proto-17 01:12:23.599657 IP 172.31.201.100.5060 > 172.31.200.100.5060: UDP, length 2000 01:12:23.599666 IP 172.31.201.100 > 172.31.200.100: ip-proto-17 01:12:24.603630 IP 172.31.201.100.5060 > 172.31.200.100.5060: UDP, length 2000 01:12:24.603639 IP 172.31.201.100 > 172.31.200.100: ip-proto-17 Into IPsec on J 01:13:29.451155 (authentic,confidential): SPI 0xc99a1e36: IP 172.31.201.100.5060 > 172.31.200.100.5060: UDP, length 2000 01:13:30.463517 (authentic,confidential): SPI 0xc99a1e36: IP 172.31.201.100.5060 > 172.31.200.100.5060: UDP, length 2000 01:13:31.482213 (authentic,confidential): SPI 0xc99a1e36: IP 172.31.201.100.5060 > 172.31.200.100.5060: UDP, length 2000 Out of IPsec on H 01:15:00.971973 (authentic,confidential): SPI 0xc99a1e36: IP 172.31.201.100.5060 > 172.31.200.100.5060: UDP, length 2000 01:15:01.997529 (authentic,confidential): SPI 0xc99a1e36: IP 172.31.201.100.5060 > 172.31.200.100.5060: UDP, length 2000 01:15:03.029548 (authentic,confidential): SPI 0xc99a1e36: IP 172.31.201.100.5060 > 172.31.200.100.5060: UDP, length 2000 Out H LAN 01:16:09.025710 IP 172.31.201.100.5060 > 172.31.202.100.5060: UDP, length 2000 01:16:09.025715 IP 172.31.201.100 > 172.31.202.100: ip-proto-17 01:16:10.047232 IP 172.31.201.100.5060 > 172.31.202.100.5060: UDP, length 2000 01:16:10.047239 IP 172.31.201.100 > 172.31.202.100: ip-proto-17 01:16:11.066202 IP 172.31.201.100.5060 > 172.31.202.100.5060: UDP, length 2000 01:16:11.066209 IP 172.31.201.100 > 172.31.202.100: ip-proto-17 (Note that ICMP unreachables were returned by 172.31.202.100 since there is nothing listening on udp/5060. They were received by nping. This verifies good end-to-end connectivity.) Disable pf on pfSense J to avoid fragment reassembly / scrubbing With pf enabled on H and disabled on J Into J LAN 01:21:32.108916 IP 172.31.201.100.5060 > 172.31.200.100.5060: UDP, length 2000 01:21:32.108982 IP 172.31.201.100 > 172.31.200.100: ip-proto-17 01:21:33.132073 IP 172.31.201.100.5060 > 172.31.200.100.5060: UDP, length 2000 01:21:33.132135 IP 172.31.201.100 > 172.31.200.100: ip-proto-17 01:21:34.153074 IP 172.31.201.100.5060 > 172.31.200.100.5060: UDP, length 2000 01:21:34.153133 IP 172.31.201.100 > 172.31.200.100: ip-proto-17 Into IPsec on J 01:22:13.844357 (authentic,confidential): SPI 0xc99a1e36: IP 172.31.201.100.5060 > 172.31.200.100.5060: UDP, length 2000 01:22:13.844409 (authentic,confidential): SPI 0xc99a1e36: IP 172.31.201.100 > 172.31.200.100: ip-proto-17 01:22:14.853298 (authentic,confidential): SPI 0xc99a1e36: IP 172.31.201.100.5060 > 172.31.200.100.5060: UDP, length 2000 01:22:14.853390 (authentic,confidential): SPI 0xc99a1e36: IP 172.31.201.100 > 172.31.200.100: ip-proto-17 01:22:15.857681 (authentic,confidential): SPI 0xc99a1e36: IP 172.31.201.100.5060 > 172.31.200.100.5060: UDP, length 2000 01:22:15.857729 (authentic,confidential): SPI 0xc99a1e36: IP 172.31.201.100 > 172.31.200.100: ip-proto-17 Out of IPsec on H 01:23:08.108706 (authentic,confidential): SPI 0xc99a1e36: IP 172.31.201.100.5060 > 172.31.200.100.5060: UDP, length 2000 01:23:08.108729 (authentic,confidential): SPI 0xc99a1e36: IP 172.31.201.100 > 172.31.200.100: ip-proto-17 01:23:09.109471 (authentic,confidential): SPI 0xc99a1e36: IP 172.31.201.100.5060 > 172.31.200.100.5060: UDP, length 2000 01:23:09.109493 (authentic,confidential): SPI 0xc99a1e36: IP 172.31.201.100 > 172.31.200.100: ip-proto-17 01:23:10.115393 (authentic,confidential): SPI 0xc99a1e36: IP 172.31.201.100.5060 > 172.31.200.100.5060: UDP, length 2000 01:23:10.115415 (authentic,confidential): SPI 0xc99a1e36: IP 172.31.201.100 > 172.31.200.100: ip-proto-17 Out H LAN [No packets captured] Note that removing the BINAT on IPsec on H changes this behavior. These captures are the same (Replacing destination with 172.31.202.100 because no more BINAT): Into J LAN Into IPsec on J Out of IPsec on H But the following is captured on H LAN 01:41:02.519981 IP 172.31.201.100.5060 > 172.31.202.100.5060: UDP, length 2000 01:41:03.524742 IP 172.31.201.100.5060 > 172.31.202.100.5060: UDP, length 2000 01:41:04.528723 IP 172.31.201.100.5060 > 172.31.202.100.5060: UDP, length 2000 Note: Not fragmented according to the interface’s mtu 1500. This is similar to what has been reported on #7779 for OpenVPN. Upgraded J to 2.4-RC - Same behavior Upgraded H to 2.4-RC - Same behavior
Files
Related issues
Updated by Renato Botelho about 7 years ago
- Target version changed from 2.4.0 to 2.4.1
Updated by Jim Pingle about 7 years ago
- Target version changed from 2.4.1 to 2.4.2
Moving target to 2.4.2 as we need 2.4.1 sooner than anticipated.
Updated by Jim Pingle about 7 years ago
- Target version changed from 2.4.2 to 2.4.3
Updated by Anonymous over 6 years ago
- Assignee changed from Jim Pingle to Luiz Souza
- Target version changed from 2.4.3 to 2.4.4
++target_version
Updated by Franciszek Koltuniuk about 6 years ago
Hi,
I have a similar issue with fragmented packets send/received over IPsec tunnel.
Finally, I manage to update /tmp/rules.debug and make traffic to be passed as expected:
- add scrub rule that apply scrubbing on a traffic from networks behind IPsec vpn:
scrub from <from_ipsec_network> to any no-df fragment reassemble
- add scrub rult that disable scrubbing on a traffic from a local interface to the network behind IPsec vpn:
no scrub on $INTERFACE to <from_ipsec_network>
So the only one thing is missing: pfsense web interface that allows to setup ipsec in the same way..
Updated by Andi Admin almost 6 years ago
Any chance to get fixed soon? This bug actually prevent our VPN from being usable for VoIP which uses UDP and in some cases need fragments.
Updated by Next Next almost 6 years ago
Hi, I also have a similar issue with fragmented packets and IPsec tunnels (noticed with ICMP traffic).
Incoming fragmented packets (either from 'LAN to IPsec' or 'IPsec to LAN') are forwarded unfragmented (packets larger then interface MTU).
Disabling scrubbing globally makes the problem go away, but this doesn't seem to be a good solution as it affects much more than only bug-related traffic.
Is there any info on what's causing this? Is it PFsense/config related or maybe something in BSD itself ?
Updated by Gabriel Latour almost 6 years ago
Hi, I have been waiting a year for that fix, for us, it's RDS sessions that disconnects randomly when using UDP over IPSEC.
Updated by Anonymous almost 4 years ago
- Target version changed from 2.5.0 to CE-Next
Updated by Rai Wol over 3 years ago
I hit the same issue with EAP-TLS (Wireless authentication) UDP fragmented packages from AP to NPS (Radius) server not arriving. After turning off scrubbing it started working. Maybe add an option to enable/disable scrubbing per interface.
Updated by Viktor Gurov over 3 years ago
Franciszek Koltuniuk wrote:
Hi,
I have a similar issue with fragmented packets send/received over IPsec tunnel.
Finally, I manage to update /tmp/rules.debug and make traffic to be passed as expected:
- add scrub rule that apply scrubbing on a traffic from networks behind IPsec vpn:
scrub from <from_ipsec_network> to any no-df fragment reassemble
- add scrub rult that disable scrubbing on a traffic from a local interface to the network behind IPsec vpn:
no scrub on $INTERFACE to <from_ipsec_network>
So the only one thing is missing: pfsense web interface that allows to setup ipsec in the same way..
related to #10493:
"Because of filter_get_vpns_list() returns not only IPsec networks, IPsec MSS clamping option will affect unnecessary VPN types"
Updated by Viktor Gurov over 3 years ago
Updated by Jim Pingle over 3 years ago
- Status changed from New to Pull Request Review
- Target version changed from CE-Next to 2.6.0
Updated by Renato Botelho over 3 years ago
- Status changed from Pull Request Review to Feedback
- Assignee changed from Luiz Souza to Viktor Gurov
- Plus Target Version set to 21.09
PR has been merged. Thanks!
Updated by Viktor Gurov over 3 years ago
- % Done changed from 0 to 100
Applied in changeset a8e97945b4fdaa9c5228bddf2964d95fb505ee4b.
Updated by Chris Linstruth over 3 years ago
- Status changed from Feedback to Assigned
The new checkboxes in System > Advanced, Firewall & NAT are not populated when re-entering the configuration page.
Checking IP Do-Not-Fragment compatibility and IP Fragment Reassemble seem to work but going back tot he config page they are unchecked again.
Enable Maximum MSS looks like it "sticks."
The oversized, fragmented packets do look like they are going from LAN host to LAN host though so that looks to be working.
Updated by Viktor Gurov over 3 years ago
Chris Linstruth wrote:
The new checkboxes in System > Advanced, Firewall & NAT are not populated when re-entering the configuration page.
Checking IP Do-Not-Fragment compatibility and IP Fragment Reassemble seem to work but going back tot he config page they are unchecked again.
Enable Maximum MSS looks like it "sticks."
fix:
https://gitlab.netgate.com/pfSense/pfSense/-/merge_requests/294
Updated by Renato Botelho over 3 years ago
- Status changed from Assigned to Feedback
Updated by Jim Pingle over 3 years ago
- Status changed from Feedback to Resolved
Updated by Marcos M about 3 years ago
- Status changed from Resolved to Feedback
I was able to test this fix and noticed there are two issues which I needed to work around in order for large df-bit-set traffic to flow correctly.
- The
vpn_networks
alias does not take into account IPsec P2s with an address type, only network. Hence, I needed to update the tunnels accordingly. - The new
IP Do-Not-Fragment compatibility
option cannot be unchecked after it's been checked and saved. To reproduce:- Check the option, then save.
- Refresh the page to verify it's still checked.
- Uncheck the option then save.
- Refresh the page. The option is now still checked.
Updated by Marcos M about 3 years ago
- Status changed from Feedback to Pull Request Review
The following merge request addresses the two issues outlined in my previous comment:
https://gitlab.netgate.com/pfSense/pfSense/-/merge_requests/356
Updated by Jim Pingle about 3 years ago
- Status changed from Pull Request Review to Feedback
PR merged.
Updated by Jim Pingle about 3 years ago
- Plus Target Version changed from 21.09 to 22.01
Updated by Viktor Gurov over 2 years ago
- Related to Bug #7779: Traffic crossing a site-to-site OpenVPN tunnel fails to fragment. added