Bug #13327
closed
Valid OpenVPN client connections rejected due to extraneous output to ovpn_auth_verify
100%
Description
OpenVPN was observed rejecting client connections that were previously accepted and had not expired. Research lead to /usr/local/sbin/ovpn_auth_verify. When using TLS, this code calls
/usr/local/bin/php-cgi -q /etc/inc/openvpn.tls-verify.php and compares the response to "OK". In my case the response received was "....OK", which did not match and caused the connection to be dropped. A log file is attached to this report.
Others have experienced the same problem, as shown in the last half of the forum discussion at: https://forum.netgate.com/topic/171706/user-auth-failed/5
Issuing the same php-cgi command manually showed that the leading dots are apparently a progress indicator for a (not very) long running process. This suggests that the problem will only impact slower or heavily loaded systems.
As a workaround I altered ovpn_auth_verify as follows as compared to git commit 8f2f85c:
@41a42
> RESULT=$(echo $RESULT | tr -d ".")
A copy of the modified ovpn_auth_verify file will be attached to this report.
I first observed and patched this problem 15/Jul/2021. I am available to test proposed changes.
Files
Updated by Jim Pingle almost 2 years ago
- Status changed from New to Rejected
There isn't enough information to go on here. This is working for us in the lab and for most if not all users of the current release.
The linked forum thread references 2.4.5 which is very outdated. We can only accept reports against the most recent release.
If you can find a way to replicate it on a clean installation of a current release, or ideally on the latest development snapshot, please provide the entire procedure to reproduce the problem.
Updated by Brian Martin almost 2 years ago
- File ovpn.cfg.sanitized ovpn.cfg.sanitized added
I neglected to mention in the bug report and the forum thread that I'm on release 2.6.0, the current stable release. Further, the affected file, ovpn_auth_verify, has not been subsequently changed from the master according to GitHub, so I'm at the very latest version of that file at least.
Regarding replicating the problem, I think the key is the hardware I'm running it on, and the fact I'm using TLS.
Here are excerpts from the pfSense dashboard that may help on the hardware side:
CPU Type Intel(R) Atom(TM) CPU C2758 @ 2.40GHz 8 CPUs: 1 package(s) x 8 core(s) AES-NI CPU Crypto: Yes (inactive) QAT Crypto: Yes (inactive) Memory: 4G Current load average: 0.14, 0.19, 0.16
So, an old system but perfectly adequate for my needs except for this one issue.
I'm using TLS-verify, which causes ovpn_verify_auth to take a different path and call /etc/inc/openvpn.tls-verify.php. I don't know enough PHP to understand the script well, but I see it calls openssl in several places, and openssl often prints dots as a progress indicator. This may be the source of the stray dots that you saw in the log attached previously.
I don't have a lab system to test on, and I'm somewhat hesitant to move off of STABLE on my production system, but I have good backups and can do that if you think that is necessary.
I'll attach a copy of my OpenVPN configuration (hopefully adequately sanitized -- please alert me if I've published anything sensitive) so you can see all my settings.
The problem occurs every time without my patch, and never occurs with the patch. Some others are seeing it too, although not very many.
I've previously offered to test patch candidates. How else can I help you reproduce the problem? I'm ready to assist in any way I can.
Updated by Massimo Vannucci almost 2 years ago
I'm experiencing the exact same problem reported by Brian Martin.
Unfortunately I don't have enough knowledge of PHP to understand why it returns a "....OK" for us, so it is difficult for me to tell you how to reprodure the steps.
pfSense version
2.6.0-RELEASE (amd64)
built on Mon Jan 31 19:57:53 UTC 2022
FreeBSD 12.3-STABLE
Hardware
Intel(R) Celeron(R) CPU J3160 @ 1.60GHz
4 CPUs: 1 package(s) x 4 core(s)
AES-NI CPU Crypto: Yes (active)
QAT Crypto: No
4GB of RAM
Let me know if there are other tests I can run to help everybody to fix this issue.
Updated by Marcos M 11 months ago
Brian Martin Do you still experience the issue on pfSense+ 23.05?
Updated by Brian Martin 11 months ago
I'm glad to hear this issue hasn't been forgotten.
I'll need some help to answer that. I'm using the community edition, and the dashboard reports I'm on the latest edition, identified as "2.6.0-RELEASE". I don't know how to correlate that to "pfSense+ 23.05".
To work around the issue I patched /usr/local/sbin/ovpn_auth_verify to insert the following after gathering RESULT but before comparing it to "OK":
RESULT=$(echo $RESULT | tr -d ".")
That patch is still there, but if I comment it out the failure still occurs.
If 2.6.0-RELEASE correlates to pfSense+ 23.05, please provide a checksum of the unaltered /usr/local/sbin/open_auth_verify, so I can verify that I am testing the correct version of the code. Otherwise, please advise me as to how to come up to the equivalent version, and I'll be happy to retest.
Updated by Marcos M 11 months ago
Migrate to pfSense+ by following the guide here:
https://docs.netgate.com/pfsense/en/latest/install/migrate-to-plus.html
Alternatively, update to pfSense CE 2.7 (System > Update).
Updated by Brian Martin 11 months ago
Thank you. CE 2.7 is still in development, and I'm not currently interested in to moving to pfSense+, so I won't be prepared to test for a bit. Once CE 2.7 reaches stable status, I expect to move to it fairly quickly. I'm sorry I can't be of any help sooner. I'd really like to test this fix.
Updated by Brian Martin 9 months ago
I just tested with CE 2.7. I confirmed that my [[patch: https://redmine.pfsense.org/issues/13327#note-5]] is no longer present after the upgrade, and that I now no longer need the patch in order to log in. The problem appears to be fixed to me. That's great! Thank you so much.
May I ask ... what changed that fixed this issue.
Updated by Jim Pingle 9 months ago
- Status changed from Rejected to Resolved
- % Done changed from 0 to 100