pfSense

I generated a configuration with 250 VLANs (assigned, enabled, with DHCP active) based on a user complaint of problems with a similar number of VLANs.

When booting up most things are OK, but slow. All the VLANs show on the console, in the widgets, menus, etc, but making a change to any one of the VLANs seems to get it stuck in a very slow cycle of touching every interface, which is disruptive and also makes the GUI sluggish and occasionally non-responsive.

See also: https://forum.netgate.com/topic/136946/dhcp-fails-silently-but-works-on-reboot-of-pfsense/

I made a change to only one VLAN, and then tried to apply the changes. As you can see from the logs, it's somehow taking almost two minutes to process each VLAN:

Nov 13 13:46:49 missy check_reload_status: Reloading filter
Nov 13 13:50:21 missy check_reload_status: Syncing firewall
Nov 13 13:50:23 missy check_reload_status: Restarting ipsec tunnels
Nov 13 13:51:38 missy check_reload_status: updating dyndns opt5
Nov 13 13:51:38 missy check_reload_status: Restarting ipsec tunnels
Nov 13 13:51:38 missy kernel: vlan0: changing name to 'ix3.1'
Nov 13 13:52:51 missy check_reload_status: updating dyndns opt5
Nov 13 13:52:51 missy kernel: vlan1: changing name to 'ix3.2'
Nov 13 13:52:51 missy check_reload_status: Restarting ipsec tunnels
Nov 13 13:53:22 missy missy.lab.jimp.pw nginx: 2018/11/13 13:53:22 [error] 16591#100204: *1 upstream timed out (60: Operation timed out) while reading response header from upstream, client: 198.51.100.108, server: , request: "POST /interfaces.php?if=opt5 HTTP/2.0", upstream: "fastcgi://unix:/var/run/php-fpm.socket", host: "missy.lab.jimp.pw", referrer: "https://missy.lab.jimp.pw/interfaces.php?if=opt5" 
Nov 13 13:54:05 missy check_reload_status: updating dyndns opt6
Nov 13 13:54:05 missy check_reload_status: Restarting ipsec tunnels
Nov 13 13:54:05 missy kernel: vlan2: changing name to 'ix3.3'
Nov 13 13:55:43 missy check_reload_status: updating dyndns opt7
Nov 13 13:55:44 missy kernel: vlan3: changing name to 'ix3.4'
Nov 13 13:55:44 missy check_reload_status: Restarting ipsec tunnels
Nov 13 13:57:41 missy check_reload_status: updating dyndns opt8
Nov 13 13:57:41 missy check_reload_status: Restarting ipsec tunnels
Nov 13 13:57:41 missy kernel: vlan4: changing name to 'ix3.5'
Nov 13 13:59:39 missy check_reload_status: updating dyndns opt9
Nov 13 13:59:40 missy kernel: vlan5: changing name to 'ix3.6'
Nov 13 13:59:40 missy check_reload_status: Restarting ipsec tunnels
Nov 13 14:01:36 missy check_reload_status: updating dyndns opt10
Nov 13 14:01:37 missy kernel: vlan6: changing name to 'ix3.7'
Nov 13 14:01:37 missy check_reload_status: Restarting ipsec tunnels
Nov 13 14:03:09 missy check_reload_status: updating dyndns opt11
Nov 13 14:03:09 missy check_reload_status: Restarting ipsec tunnels
Nov 13 14:03:09 missy kernel: vlan7: changing name to 'ix3.8'

The contents of /tmp/.interfaces.apply also agree that only one interface was touched:

a:1:{s:4:"opt5";a:2:{s:5:"ifcfg";a:7:{s:5:"descr";s:3:"VL1";s:2:"if";s:5:"ix3.1";s:6:"enable";s:0:"";s:6:"ipaddr";s:10:"198.18.1.1";s:6:"subnet";s:2:"24";s:8:"spoofmac";s:0:"";s:6:"realif";s:5:"ix3.1";}s:4:"ppps";a:0:{}}}

Occasionally a PHP thread will go up to 100% CPU most most of the time is spent idle/waiting.

This is on an SG-5100, but the customer in the thread above saw it on an XG-1541.