Project

General

Profile

Actions

Bug #11626

closed

Google LDAP connections fail due to lack of SNI for TLS 1.3

Added by Viktor Gurov about 3 years ago. Updated about 1 year ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Authentication
Target version:
Start date:
Due date:
% Done:

100%

Estimated time:
Release Notes:
Default
Affected Plus Version:
21.02
Affected Architecture:

Description

https://forum.netgate.com/topic/161725/google-ldap-connection-failed:

I have a problem after update my Netgate XG-7100 to the version 21.02-RELEASE-p1.
After the update my pfSense failed to bind to ldap.google.com.

Attempting connection to ldap.google.com OK
Attempting bind to ldap.google.com failed

openldap24-client doesn't support SNI:

# ldapsearch -H ldaps://ldap.google.com -x -d1
ldap_url_parse_ext(ldaps://ldap.google.com)
ldap_create
ldap_url_parse_ext(ldaps://ldap.google.com:636/??base)
ldap_sasl_bind
ldap_send_initial_request
ldap_new_connection 1 1 0
ldap_int_open_connection
ldap_connect_to_host: TCP ldap.google.com:636
ldap_new_socket: 3
ldap_prepare_socket: 3
ldap_connect_to_host: Trying 216.239.32.58:636
ldap_pvt_connect: fd: 3 tm: -1 async: 0
attempting to connect: 
connect success
TLS trace: SSL_connect:before SSL initialization
TLS trace: SSL_connect:SSLv3/TLS write client hello
TLS trace: SSL_connect:SSLv3/TLS write client hello
TLS trace: SSL_connect:SSLv3/TLS read server hello
TLS trace: SSL_connect:TLSv1.3 read encrypted extensions
TLS trace: SSL_connect:SSLv3/TLS read server certificate request
TLS certificate verification: depth: 0, err: 18, subject: /OU=No SNI provided; please fix your client./CN=invalid2.invalid, issuer: /OU=No SNI provided; please fix your client./CN=invalid2.invalid
TLS certificate verification: Error, self signed certificate
TLS trace: SSL3 alert write:fatal:unknown CA
TLS trace: SSL_connect:error in error
TLS: can't connect: error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed (self signed certificate).
ldap_err2string
ldap_sasl_bind(SIMPLE): Can't contact LDAP server (-1)

https://support.google.com/a/answer/9190869?hl=en:

Verify return code: 18 (self signed certificate)

The Secure LDAP service requires a TLS client that supports and initiates a TLS session using SNI (Server Name Indication). If the TLS client does not support SNI, then the TLS server (ldap.google.com) returns a self-signed certificate that will not pass CA validation checks, to indicate that SNI is required.

This behavior can be confirmed by checking the OpenSSL client output for the following line near the start of the output:

depth=0 OU = "No SNI provided; please fix your client.", CN = invalid2.invalid

Causes for this error may include an OpenSSL version that does not support SNI, or an application that uses the OpenSSL library with SNI explicitly disabled.

PHP issue: https://stackoverflow.com/questions/65885108/sni-issues-connecting-to-google-ldap-server
ldapsearch issue: https://www.openldap.org/lists/openldap-bugs/202002/msg00421.html


Files

clipboard-202212211513-cwaqi.png (10.4 KB) clipboard-202212211513-cwaqi.png Jim Pingle, 12/21/2022 02:13 PM
Actions #1

Updated by Viktor Gurov about 3 years ago

may be related to #9417

Actions #2

Updated by Jim Pingle about 3 years ago

If OpenLDAP ldapsearch fails directly it's unlikely to be related to #9417

All the references I see to SNI seem fairly recent, but I'm not sure if they changed something on the Google side or if something on the client side changed (e.g. now using TLS 1.3 where it couldn't before).

Actions #4

Updated by Jim Pingle about 3 years ago

  • Project changed from pfSense to pfSense Plus
  • Subject changed from google LDAP connection failed to Google LDAP connection failed due to lack of SNI for TLS 1.3
  • Category changed from Authentication to Authentication
  • Affected Version deleted (2.5.0)
  • Affected Plus Version set to 21.02

Not that I like the idea of downgrading to a lower TLS version but I wonder if it would work if we forced off TLS 1.3 in this case using something like this:

ldap_set_option(NULL, LDAP_OPT_X_TLS_CIPHER_SUITE, "NORMAL:!VERS-TLS1.3");

I haven't tried that option so I'm not sure if it's one that needs NULL in that first spot like the cert setup or if it can take $ldap for per-connection options.

If that does work we could add a GUI knob to disable TLS 1.3 for LDAP, maybe only when sending a client cert.

Actions #5

Updated by Alders Watne almost 3 years ago

Theoretically that would be the fix (forcing TLSv1.2 to bypass the SNI TLS v1.3 requirement), but setting this LDAP option or even with the old `putenv('LDAPTLS_CIPHER_SUITE=NORMAL:!VERS-TLS1.3')` doesn't work.

Same with `LDAPTLS_CIPHER_SUITE=NORMAL:!VERS-TLS1.3 ldapsearch -H ldaps://ldap.google.com -x -d1`.

Any ideas?

Jim Pingle wrote:

Not that I like the idea of downgrading to a lower TLS version but I wonder if it would work if we forced off TLS 1.3 in this case using something like this:

[...]

I haven't tried that option so I'm not sure if it's one that needs NULL in that first spot like the cert setup or if it can take $ldap for per-connection options.

If that does work we could add a GUI knob to disable TLS 1.3 for LDAP, maybe only when sending a client cert.

Actions #6

Updated by Jim Pingle almost 3 years ago

It would either be this:

ldap_set_option(NULL, LDAP_OPT_X_TLS_CIPHER_SUITE, "NORMAL:!VERS-TLS1.3");

Or this:

ldap_set_option($ldap, LDAP_OPT_X_TLS_CIPHER_SUITE, "NORMAL:!VERS-TLS1.3");

And likely set just before or after the places LDAP_OPT_X_TLS_REQUIRE_CERT is set in /etc/inc/auth.inc.

Actions #7

Updated by Alders Watne almost 3 years ago

The error self-signed error is gone but the bind still is unsuccessful. Same config ported over the 2.4 release line. Nothing specific about the error in the logs.

Jim Pingle wrote:

It would either be this:

[...]

Or this:

[...]

And likely set just before or after the places LDAP_OPT_X_TLS_REQUIRE_CERT is set in /etc/inc/auth.inc.

Actions #8

Updated by Michael Mogren over 2 years ago

This issue open 5 months and still no fix. The stunnel workaround is not a good solution - it only works for gui and does not work for ssh login and requires additional package/configuration. Please fix.

Actions #9

Updated by Philipp Wagner over 2 years ago

I have the same issue when setting up Google's "Secure LDAP" as user directory and did a bit more digging. Here's my take on it.

So the problem that we see today is a result of an OpenSSL version that supports TLS 1.3 and a OpenLDAP version that doesn't support SNI.

The PHP code in /etc/inc/auth.inc looks correct to me.

The real solution is to update OpenLDAP to version 2.5. I have confirmed locally that this solves the problem.

Until that happens, there are some workarounds, with of course rather different (security) properties:

What does not work is setting ldap_set_option($ldap, LDAP_OPT_X_TLS_CIPHER_SUITE, "NORMAL:!VERS-TLS1.3");. This syntax is for GnuTLS, but pfSense has built OpenLDAP with OpenSSL. OpenSSL does not seem to have a way to disable TLS 1.3 in a similar fashion as GnuTLS, see also https://mta.openssl.org/pipermail/openssl-users/2019-March/010150.html.

Finally, if you're looking for a reasonably easy to reproduce the problem with more log information take the following script, save it to your pfSense machine, and run it on the terminal (e.g. sudo php test.php).

<?php
ldap_set_option(NULL, LDAP_OPT_DEBUG_LEVEL, 255);

$ds=ldap_connect("ldaps://ldap.google.com:636"); 
# There will be certs in the directory after you've set up Google Secure LDAP in the UI. Adjust the filename.
ldap_set_option(NULL, LDAP_OPT_X_TLS_CERTFILE, "/var/run/certs/your-cert.crt");
ldap_set_option(NULL, LDAP_OPT_X_TLS_KEYFILE, "/var/run/certs/your-cert.key");
ldap_set_option($ds, LDAP_OPT_PROTOCOL_VERSION, 3);
ldap_set_option($ds, LDAP_OPT_REFERRALS, 0);

$re = ldap_start_tls($ds);
if (!$re) {
   echo ldap_error($ldap);
   ldap_get_option($ldap, LDAP_OPT_DIAGNOSTIC_MESSAGE, $err);
   echo $err;
}

echo "connected!";

$results = ldap_search($ldap, 'dc=example,dc=com', 'uid=*');
print_r(ldap_get_entries($ldap, $results));

ldap_close($ds);

Here's how it looks for me:

$ sudo php test.php 
ldap_url_parse_ext(ldap://localhost/)
ldap_init: trying /usr/local/etc/openldap/ldap.conf
ldap_init: using /usr/local/etc/openldap/ldap.conf
ldap_init: HOME env is /root
ldap_init: trying /root/ldaprc
ldap_init: trying /root/.ldaprc
ldap_init: trying ldaprc
ldap_init: LDAPCONF env is NULL
ldap_init: LDAPRC env is NULL
ldap_create
ldap_url_parse_ext(ldaps://ldap.google.com:636)
ldap_extended_operation_s
ldap_extended_operation
ldap_send_initial_request
ldap_new_connection 1 1 0
ldap_int_open_connection
ldap_connect_to_host: TCP ldap.google.com:636
ldap_new_socket: 11
ldap_prepare_socket: 11
ldap_connect_to_host: Trying 216.239.32.58:636
ldap_pvt_connect: fd: 11 tm: -1 async: 0
attempting to connect: 
connect success
TLS trace: SSL_connect:before SSL initialization
TLS trace: SSL_connect:SSLv3/TLS write client hello
TLS trace: SSL_connect:SSLv3/TLS write client hello
TLS trace: SSL_connect:SSLv3/TLS read server hello
TLS trace: SSL_connect:TLSv1.3 read encrypted extensions
TLS trace: SSL_connect:SSLv3/TLS read server certificate request
TLS certificate verification: depth: 0, err: 18, subject: /OU=No SNI provided; please fix your client./CN=invalid2.invalid, issuer: /OU=No SNI provided; please fix your client./CN=invalid2.invalid
TLS certificate verification: Error, self signed certificate
TLS trace: SSL3 alert write:fatal:unknown CA
TLS trace: SSL_connect:error in error
TLS: can't connect: error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed (self signed certificate).
ldap_err2string
Actions #10

Updated by Jim Pingle over 2 years ago

Thanks for tracking that down!

The main problem now is that OpenLDAP 2.5 isn't currently available in the FreeBSD ports tree, which is a blocker. I'm not sure why, as it's been around for a while.

Either the OpenLDAP port maintainer will need to add it, or someone will need to submit the new set of ports for OpenLDAP 2.5 to FreeBSD.

Actions #11

Updated by Viktor Gurov over 2 years ago

Philipp Wagner wrote in #note-9:

Does this work if you manually replace LDAP_OPT_X_TLS_HARD with LDAP_OPT_X_TLS_NEVER in auth.inc?

Actions #12

Updated by Kris Phillips over 2 years ago

It appears that openldap25-client and openldap25-server are both in freshports for FreeBSD.

https://www.freshports.org/net/openldap25-client/

Actions #13

Updated by Jim Pingle over 2 years ago

  • Assignee set to Renato Botelho
  • Target version set to 22.01
Actions #14

Updated by Jim Pingle about 2 years ago

  • Assignee changed from Renato Botelho to Luiz Souza

Looks like the ports tree is setup to where we can define the version to use without much fuss. It may be as simple as this change before building:

diff --git a/tools/conf/pfPorts/make.conf b/tools/conf/pfPorts/make.conf
index 2f4a9bf9ca..5358cb5247 100644
--- a/tools/conf/pfPorts/make.conf
+++ b/tools/conf/pfPorts/make.conf
@@ -28,6 +28,9 @@ PHP_FD_SETSIZE=               3172
 WITH_DEBUG=    yes
 . endif

+# Use OpenLDAP 2.5
+WITH_OPENLDAP_VER=25
+
 # Options for packages

 # databases
Actions #15

Updated by Jim Pingle about 2 years ago

  • Target version changed from 22.01 to 22.05

Nudge this ahead so we have more time to ensure there aren't any regressions from the change.

Actions #16

Updated by Jim Pingle almost 2 years ago

  • Target version changed from 22.05 to 23.01
Actions #17

Updated by Kris Phillips over 1 year ago

Jim Pingle wrote in #note-15:

Nudge this ahead so we have more time to ensure there aren't any regressions from the change.

Jim,

Can we get this into 22.09 builds after 22.05 is out? This is a common issue people are running into more and more.

Actions #18

Updated by Jim Pingle over 1 year ago

Not up to me, it'll need to be handled by Luiz or Brad once things start moving for 22.09 but it's already on the radar.

Actions #19

Updated by Kris Phillips over 1 year ago

Builds for 23.01 are including openldap26-client version 2.6.3, so this can probably be marked as Feedback.

Actions #20

Updated by Jim Pingle over 1 year ago

  • Status changed from New to Feedback
  • Start date deleted (03/05/2021)
  • % Done changed from 0 to 100
Actions #21

Updated by Jim Pingle over 1 year ago

  • Subject changed from Google LDAP connection failed due to lack of SNI for TLS 1.3 to Google LDAP connections fail due to lack of SNI for TLS 1.3

Updating subject for release notes.

Actions #22

Updated by Jim Pingle about 1 year ago

We don't have an account with Google LDAP auth any longer, but I can confirm that the LDAP client is sending the SNI header during TLS negotiation which was the required point here.

Actions

Also available in: Atom PDF