local_sync_accounts() slowness can trigger GUI/XMLRPC failures with many accounts
When a firewall has many local accounts, the time it takes for local_sync_accounts() to finish grows large enough to trigger timeouts and other problems for XMLRPC.
Notably, in an HA cluster this becomes a burden because that function is called for each filter sync.
On stand-alone firewalls the function is only called during bootup, so it does not have quite the same impact in that scenario, though it still delays the boot process.
To illustrate the issue I made a playback script that calls local_sync_accounts() and nothing else:
: grep -c '<user>' /conf/config.xml 44 : time pfSsh.php playback localsyncusers 1.379u 5.649s 1:17.53 9.0% 633+216k 0+16190io 0pf+0w
For 44 accounts this particular test firewall needed 1 min 17 seconds to complete the sync process. This could easily overrun PHP/XMLRPC timeouts depending on the speed of the firewall cpu/disks/etc.
We have at least one customer hitting the issue ( 16693 ), plus at least one user report ( https://forum.pfsense.org/index.php?topic=127546.0 )
- Rename local_sync_accounts() to local_reset_accounts() and keep it
only being used /etc/rc.bootup
- Reimplement local_sync_accounts() receiving a list of users and
groups to be added and/or deleted
- Remove call to filter_configure xmlrpc method from
rc.filter_synchronize since it's now called internally from
- On restore_config_section implementation stop copying all content
from user/group sections. Instead check for new/modified/deleted
items and create necessary arrays to be passed to local_syng_accounts
- Add a parameter to filter_configure xmlrpc method to decide when to
call a full reset of users/groups using local_reset_accounts()