Project

General

Profile

Bug #6367

Long delays with LDAP enabled w/local users during boot at "Synchronizing user settings..."

Added by Brett Merrick over 1 year ago. Updated 6 months ago.

Status:
Resolved
Priority:
High
Category:
User manager
Target version:
Start date:
05/18/2016
Due date:
% Done:

100%

Affected version:
2.3.1
Affected Architecture:

Description

This occurs when:
  • an LDAP authentication server is selected under User Manager > Settings > Authentication Server
  • the selected LDAP server is configured using a FQDN rather than an IP address
  • the FQDN of the LDAP server is not configured as an override in the DNS resolver and therefore is not in the /etc/hosts file.

The userlog show a delay of around 5 minutes per user:

2016-05-19 10:58:21 [unknown:usermod] root(0):wheel(0):Charlie &:/root:/bin/sh
2016-05-19 10:58:21 [unknown:useradd] admin(0):wheel(0):System Administrator:/root:/etc/rc.initial
2016-05-19 10:58:21 [unknown:useradd] admin(0) home /root made
2016-05-19 11:03:23 [unknown:useradd] xxxxx(2001):nobody(65534):X XXXX:/home/xxxxx:/sbin/nologin
2016-05-19 11:03:23 [unknown:useradd] xxxxx(2001) home /home/xxxxx made
2016-05-19 11:08:23 [unknown:useradd] yyyyy(2002):nobody(65534):Y YYYY:/home/yyyyy:/sbin/nologin
2016-05-19 11:08:23 [unknown:useradd] yyyyy(2002) home /home/yyyyy made
2016-05-19 11:13:24 [unknown:useradd] zzzzz(2003):nobody(65534):Z ZZZZ:/home/zzzzz:/sbin/nologin
2016-05-19 11:13:24 [unknown:useradd] zzzzz(2003) home /home/zzzzz made
...


This seemed to start immediately after an upgrade from 2.3.0 to 2.3.1, however I cannot see any difference in the relevant startup order between these versions so I am unsure why I haven't encountered it previously.

It would make sense however to have the DNS resolver available before any code involving DNS queries.

ie. could/should rc.bootup run these

/* start dnsmasq service */
services_dnsmasq_configure();

/* start unbound service */
services_unbound_configure();

ahead of this:

echo "Synchronizing user settings...";
local_sync_accounts();

Associated revisions

Revision e8c09a23
Added by Chris Buechler over 1 year ago

Lower default LDAP timeout to 5 seconds. Idea from Sandeep1991 in PR 2971. Ticket #6367

Revision 45859aae
Added by Chris Buechler over 1 year ago

Lower default LDAP timeout to 5 seconds. Idea from Sandeep1991 in PR 2971. Ticket #6367

Revision dd4053d5
Added by Chris Buechler over 1 year ago

Lower default LDAP timeout to 5 seconds. Idea from Sandeep1991 in PR 2971. Ticket #6367

History

#1 Updated by Sandeep K V over 1 year ago

Though LDAP is causing this delay, I do not think it is due to this as we have the same rc.bootup file for long time. This is my understanding - if it was what was causing the delay you must have had faced the similar scenario back before upgrade. I think it is due to LDAP timeout I believe. Let me see!

#2 Updated by Chris Buechler over 1 year ago

  • Subject changed from System appears to hang on startup at "Synchronizing user settings..." to Long delays with LDAP enabled w/local users during boot at "Synchronizing user settings..."
  • Category set to User manager
  • Status changed from New to Confirmed
  • Target version set to 2.3.2

This came about because I fixed #6352, so it's now doing things it should have been doing but wasn't previously in the group modification. Because your auth's set to LDAP, it tries to connect at that point when it's obtaining user permissions (though for syncing local users, that's unnecessary).

Reducing the LDAP timeout helps some, but that alone is inadequate. The loop it goes through hits that timeout multiple times per user depending on config (looks like 12 times in Brett's case), so even reducing the timeout to 5 seconds would still leave a 1 minute delay per user there with Brett's config. I went ahead and committed that though, since it does help, and 25 seconds is an excessive delay there.

2.3.1_1 will have the reduced LDAP timeout. A proper fix, to not do LDAP lookups while creating the local user database (since it's not relevant there) will require more work and testing.

#3 Updated by Chris Buechler about 1 year ago

  • Target version changed from 2.3.2 to 2.4.0

#4 Updated by Jim Thompson 10 months ago

  • Assignee set to Renato Botelho

#5 Updated by Renato Botelho 10 months ago

  • Status changed from Confirmed to Feedback
  • % Done changed from 0 to 100

I've pushed a fix for #6857 that should fix it

#6 Updated by Jim Pingle 9 months ago

  • Status changed from Feedback to Resolved

Works, no delay on boot with LDAP enabled when local accounts sync

#7 Updated by Ilya Kogan 6 months ago

I hate to comment on an old issue but I couldn't find one for the "proper fix" as mentioned above. Today my LDAP server ran out of open files and, rather than simply not accept connections, connections would hang. This caused the web GUI to freeze up and, upon rebooting pfSense, would cause it to get stuck at "Synchronizing user settings...". You can imagine that it's not particularly desirable to have a working network connection and LDAP server just for a network appliance to boot.

Is this something that can be done in the background maybe and not hold up the boot process with some very strong timeouts that can withstand an LDAP server hanging the TCP connection indefinitely?

Also available in: Atom PDF