Project

General

Profile

Bug #11575

OpenVPN clients cannot pass traffic when reconnecting using the same source port

Added by Jim Pingle about 2 months ago. Updated 3 days ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
OpenVPN
Target version:
Start date:
02/28/2021
Due date:
% Done:

0%

Estimated time:
Affected Version:
Affected Architecture:
Release Notes:
Default

Description

If an OpenVPN client reconnects immediately after disconnecting, in certain cases it cannot pass traffic.

This appears to be related to the client using the same port when reconnecting. Adding nobind or lport 0 to the client configuration allows it to function as expected. This is viable for remote access clients but not all site-to-site connections which may need to use the same source port in certain cases. We do already default to lport 0 for clients on the firewall.

Initial testing doesn't show it as related to pf states or similar, since clearing the states does not affect this condition.

Since it seems to be a problem in OpenVPN itself, this is mostly for tracking and checking future versions of OpenVPN to see if they solve the problem.

History

#1 Updated by Jim Pingle about 2 months ago

  • Subject changed from OpenVPN clients cannot pass traffic when reconnecting to OpenVPN clients cannot pass traffic when reconnecting using the same source port

OpenVPN 2.5.1 does not appear to make a difference for this. I built a package for FreeBSD and loaded it, as well as updating a client, and the same behavior happened there.

#2 Updated by IT Support about 2 months ago

adding nobind fixes the problems with viscosity on mac big sur not reconnecting after a disconnect. It continues to work after 60 seconds as well, which was another issue where it timed out.

it is under the options tab as a checkbox.

#3 Updated by Edgardo Rodriguez about 1 month ago

Jim Pingle wrote:

If an OpenVPN client reconnects immediately after disconnecting, in certain cases it cannot pass traffic.

This appears to be related to the client using the same port when reconnecting. Adding nobind or lport 0 to the client configuration allows it to function as expected. This is viable for remote access clients but not all site-to-site connections which may need to use the same source port in certain cases. We do already default to lport 0 for clients on the firewall.

Initial testing doesn't show it as related to pf states or similar, since clearing the states does not affect this condition.

Since it seems to be a problem in OpenVPN itself, this is mostly for tracking and checking future versions of OpenVPN to see if they solve the problem.

I am also hit by this.
This is what better describes my scenario: https://community.openvpn.net/openvpn/ticket/1388
Enabling debug this is what I see on reconnect:
client/1.1.1.11:1194 Key [AF_INET]1.1.1.11:1194 [0] not initialized (yet), dropping packet.

I am sure by now that this is not directly related to any PFSense firewalling or any other custom thing, this is indeed related to OpenVPN

Adding lport 0 is a valid workaround (src ip:port changes and so client is "brand new"), but for short term. I would be great if renegotiation can occur without a full exchange.

#4 Updated by Edgardo Rodriguez about 1 month ago

I found that, using tcp server mode reconnection works as expected (without needing to set lport 0, or nobind, or any additional parameters on clients configs)
Also, rolled back to pfSense 2.4.4 with OpenVPN 2.4.6 and this works as expected despite the transport used. It doesn't even matters if connecting frpm OpenVPN Client 2.5.1 => OpenVPN Server 2.4.6.

I suspect about enable_async_push being now enabled at compilation time, as follows:

[2.5.0-RELEASE]
OpenVPN 2.5.0 amd64-portbld-freebsd12.2 [SSL (OpenSSL)] [LZO] [LZ4] [MH/RECVDA] [AEAD] built on Feb 5 2021
library versions: OpenSSL 1.1.1i-freebsd 8 Dec 2020, LZO 2.10
Originally developed by James Yonan
Copyright (C) 2002-2018 OpenVPN Inc <>
Compile time defines: enable_async_push=yes enable_comp_stub=no enable_crypto_ofb_cfb=yes enable_debug=yes enable_def_auth=yes enable_dlopen=unknown enable_dlopen_self=unknown enable_dlopen_self_static=unknown enable_fast_install=needless enable_fragment=yes enable_iproute2=no enable_libtool_lock=yes enable_lz4=yes enable_lzo=yes enable_management=yes enable_multihome=yes enable_pam_dlopen=no enable_pedantic=no enable_pf=yes enable_pkcs11=no enable_plugin_auth_pam=yes enable_plugin_down_root=yes enable_plugins=yes enable_port_share=yes enable_selinux=no enable_shared=yes enable_shared_with_static_runtimes=no enable_silent_rules=no enable_small=no enable_static=yes enable_strict=yes enable_strict_options=no enable_systemd=no enable_werror=no enable_win32_dll=yes enable_x509_alt_username=yes with_aix_soname=aix with_crypto_library=openssl with_gnu_ld=yes with_mem_check=no with_sysroot=no

[2.4.4-RELEASE]
OpenVPN 2.4.6 amd64-portbld-freebsd11.2 [SSL (OpenSSL)] [LZO] [LZ4] [MH/RECVDA] [AEAD] built on Oct 3 2018
library versions: OpenSSL 1.0.2o-freebsd 27 Mar 2018, LZO 2.10
Originally developed by James Yonan
Copyright (C) 2002-2018 OpenVPN Inc <>
Compile time defines: enable_async_push=no enable_comp_stub=no enable_crypto=yes enable_crypto_ofb_cfb=yes enable_debug=yes enable_def_auth=yes enable_dlopen=unknown enable_dlopen_self=unknown enable_dlopen_self_static=unknown enable_fast_install=needless enable_fragment=yes enable_iproute2=no enable_libtool_lock=yes enable_lz4=yes enable_lzo=yes enable_management=yes enable_multihome=yes enable_pam_dlopen=no enable_pedantic=no enable_pf=yes enable_pkcs11=no enable_plugin_auth_pam=yes enable_plugin_down_root=yes enable_plugins=yes enable_port_share=yes enable_selinux=no enable_server=yes enable_shared=yes enable_shared_with_static_runtimes=no enable_silent_rules=no enable_small=no enable_static=yes enable_strict=yes enable_strict_options=no enable_systemd=no enable_werror=no enable_win32_dll=yes enable_x509_alt_username=no with_aix_soname=aix with_crypto_library=openssl with_gnu_ld=yes with_mem_check=no with_sysroot=no

#5 Updated by Edgardo Rodriguez about 1 month ago

Well, confirmed what I stated before,
enable_async_push=yes breaks reconnect process when using server with UDP as transport.
Added in this commit: https://github.com/pfsense/FreeBSD-ports/commit/076aa6a7375a5867e52c98b40ce163ef0f382c5e

So, I Updated to 2.5.1 from official FreeBSD 12's repository, which shows that this flag is turned off at compilation time, like OpenVPN defaults.
https://freebsd.pkgs.org/12/freebsd-aarch64/openvpn-2.5.1.txz.html
Steps (temporal, until pfSense package gets updated) were as follows:
(dependency included: pkg: Missing dependency 'easy-rsa')
pkg add https://pkg.freebsd.org/FreeBSD:12:amd64/quarterly/All/easy-rsa-3.0.8.txz
pkg add -f https://pkg.freebsd.org/FreeBSD:12:amd64/quarterly/All/openvpn-2.5.1.txz

pkg add https://pkg.freebsd.org/FreeBSD:12:amd64/quarterly/All/easy-rsa-3.0.8.txz
Fetching easy-rsa-3.0.8.txz: 100%   44 KiB  45.4kB/s    00:01
Installing easy-rsa-3.0.8...
Extracting easy-rsa-3.0.8: 100%

pkg add -f https://pkg.freebsd.org/FreeBSD:12:amd64/quarterly/All/openvpn-2.5.1.txz
Fetching openvpn-2.5.1.txz: 100%  530 KiB 542.9kB/s    00:01
Installing openvpn-2.5.1...
package openvpn is already installed, forced install
Extracting openvpn-2.5.1: 100%
=====
Message from openvpn-2.5.1:

--
Edit /etc/rc.conf[.local] to start OpenVPN automatically at system
  startup. See /usr/local/etc/rc.d/openvpn for details.

  Connect to VPN server as a client with this command to include
  the client.up/down scripts in the initialization:
  openvpn-client <spec>.ovpn

  For compatibility notes when interoperating with older OpenVPN
  versions, please see <http://openvpn.net/relnotes.html>

  Note that OpenVPN does not officially support LibreSSL.

openvpn --version
OpenVPN 2.5.1 amd64-portbld-freebsd12.2 [SSL (OpenSSL)] [LZO] [LZ4] [MH/RECVDA] [AEAD] built on Feb 26 2021
library versions: OpenSSL 1.1.1i-freebsd  8 Dec 2020, LZO 2.10
Originally developed by James Yonan
Copyright (C) 2002-2018 OpenVPN Inc <sales@openvpn.net>
Compile time defines: enable_async_push=no enable_comp_stub=no enable_crypto_ofb_cfb=yes enable_debug=yes enable_def_auth=yes enable_dlopen=unknown enable_dlopen_self=unknown enable_dlopen_self_static=unknown enable_fast_install=needless enable_fragment=yes enable_iproute2=no enable_libtool_lock=yes enable_lz4=yes enable_lzo=yes enable_management=yes enable_multihome=yes enable_pam_dlopen=no enable_pedantic=no enable_pf=yes enable_pkcs11=no enable_plugin_auth_pam=yes enable_plugin_down_root=yes enable_plugins=yes enable_port_share=yes enable_selinux=no enable_shared=yes enable_shared_with_static_runtimes=no enable_silent_rules=no enable_small=no enable_static=yes enable_strict=yes enable_strict_options=no enable_systemd=no enable_werror=no enable_win32_dll=yes enable_x509_alt_username=no with_aix_soname=aix with_crypto_library=openssl with_gnu_ld=yes with_mem_check=no with_sysroot=no

Restarted service and proceed with testing the exact same way as before, multiple reconnects (either from client or from server having explicit-exit-notify 1), and reconnection process goes through flawlessly (only 2 icmp echo lost).

I conclude then that this flag, for openvpn pkg, should be left in it's default value which is no.

#6 Updated by Pippin MMD about 1 month ago

Asked on #openvpn-devel, this patch should fix this ticket:
https://patchwork.openvpn.net/patch/1550/

It is not related to: "enable_async_push=yes"

#7 Updated by Edgardo Rodriguez about 1 month ago

Pippin MMD wrote:

Asked on #openvpn-devel, this patch should fix this ticket:
https://patchwork.openvpn.net/patch/1550/

It is not related to: "enable_async_push=yes"

Why do you think, that my problem is automatically solved then? I don't get it

#8 Updated by Wesley Lucio dos Santos 30 days ago

Pippin MMD wrote:

Asked on #openvpn-devel, this patch should fix this ticket:
https://patchwork.openvpn.net/patch/1550/

It is not related to: "enable_async_push=yes"

Pippin MMD wrote:

Asked on #openvpn-devel, this patch should fix this ticket:
https://patchwork.openvpn.net/patch/1550/

It is not related to: "enable_async_push=yes"

Hi guys!!

Is this fix scheduled to be released in the next update?

https://redmine.pfsense.org/versions/61

#9 Updated by Jim Pingle 29 days ago

  • Target version set to CE-Next

We haven't evaluated that patch yet, but it's unlikely to make it into the next release this late in the process. If OpenVPN puts out a new patch release with it in the next couple days, perhaps we could consider that, but even that's not ideal since we wouldn't have a lot of time to ensure it is stable.

#10 Updated by Edgardo Rodriguez 29 days ago

Jim Pingle wrote:

We haven't evaluated that patch yet, but it's unlikely to make it into the next release this late in the process. If OpenVPN puts out a new patch release with it in the next couple days, perhaps we could consider that, but even that's not ideal since we wouldn't have a lot of time to ensure it is stable.

Jim, do you have any clue of why I've got this issue solved by installing a version with enable_async_push disabled ? WITHOUT the patch we are discussing here? Thanks

#11 Updated by Jim Pingle 29 days ago

No, but since you compiled it on a different system and nobody else had replicated it, it's unlikely to be related without additional confirmation.

#12 Updated by Edgardo Rodriguez 29 days ago

Jim Pingle wrote:

No, but since you compiled it on a different system and nobody else had replicated it, it's unlikely to be related without additional confirmation.

I did not actually compiled it, as I mentioned below I just switched to official FreeBSD package! The only difference though, beside this flag left as default, is the version switch 2.5.0 > 2.5.1 which has a few fixes on it.

If it were possible for me I would compile my self the same pkg with the flag switched off. Why do we have on pfSense async_push enabled?

#13 Updated by Yuran Yastreb 27 days ago

Edgardo Rodriguez wrote:

Jim Pingle wrote:

No, but since you compiled it on a different system and nobody else had replicated it, it's unlikely to be related without additional confirmation.

I did not actually compiled it, as I mentioned below I just switched to official FreeBSD package! The only difference though, beside this flag left as default, is the version switch 2.5.0 > 2.5.1 which has a few fixes on it.

If it were possible for me I would compile my self the same pkg with the flag switched off. Why do we have on pfSense async_push enabled?

I installed the packages from the repository (with your example), but I still have this problem. So don't be so categorical.

#14 Updated by Edgardo Rodriguez 27 days ago

Yuran Yastreb wrote:

Edgardo Rodriguez wrote:

Jim Pingle wrote:

No, but since you compiled it on a different system and nobody else had replicated it, it's unlikely to be related without additional confirmation.

I did not actually compiled it, as I mentioned below I just switched to official FreeBSD package! The only difference though, beside this flag left as default, is the version switch 2.5.0 > 2.5.1 which has a few fixes on it.

If it were possible for me I would compile my self the same pkg with the flag switched off. Why do we have on pfSense async_push enabled?

I installed the packages from the repository (with your example), but I still have this problem. So don't be so categorical.

Hi, English is not my born language, I am sorry If anything I said sounded rude or "categorical" as you stated.
I am trying to understand, how that patch can make things better.

I did actually solved my issue by doing what I said in comment 5. But I have no concrete answer of why, just saw that difference. What I can tell is that my issue just began after 2.4.5>2.5.0 migration (of PFsense).
Can you tell us which is your current scenario? So we can share experiences and compare evenly?

Thanks and once again, sorry If anybody got offended.

Thanks!

#15 Updated by Jason B 3 days ago

I can confirm that after upgrading our Netgate XG-7100 from 2.4.5p1 to 21.02.1 this issue began.

Neither the OpenVPN clients (nor the viscosity ones for our Mac users) had not been upgraded, however I connect the same OpenVPN clients to a pfSense box running 2.4.5 this problem does not occur.

This only seems to happen once we have upgraded to 21.02.1.

The workround of adding the following to the clients does fix the issue:
lport 0
explicit-exit-notify

Thanks

J

Also available in: Atom PDF