Project

General

Profile

Bug #9115

A large number of VLANs causes PHP issues when making an interface change

Added by Jim Pingle 6 months ago. Updated 6 months ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Interfaces
Target version:
Start date:
11/13/2018
Due date:
% Done:

100%

Estimated time:
Affected Version:
2.4.4
Affected Architecture:
All

Description

I generated a configuration with 250 VLANs (assigned, enabled, with DHCP active) based on a user complaint of problems with a similar number of VLANs.

When booting up most things are OK, but slow. All the VLANs show on the console, in the widgets, menus, etc, but making a change to any one of the VLANs seems to get it stuck in a very slow cycle of touching every interface, which is disruptive and also makes the GUI sluggish and occasionally non-responsive.

See also: https://forum.netgate.com/topic/136946/dhcp-fails-silently-but-works-on-reboot-of-pfsense/

I made a change to only one VLAN, and then tried to apply the changes. As you can see from the logs, it's somehow taking almost two minutes to process each VLAN:

Nov 13 13:46:49 missy check_reload_status: Reloading filter
Nov 13 13:50:21 missy check_reload_status: Syncing firewall
Nov 13 13:50:23 missy check_reload_status: Restarting ipsec tunnels
Nov 13 13:51:38 missy check_reload_status: updating dyndns opt5
Nov 13 13:51:38 missy check_reload_status: Restarting ipsec tunnels
Nov 13 13:51:38 missy kernel: vlan0: changing name to 'ix3.1'
Nov 13 13:52:51 missy check_reload_status: updating dyndns opt5
Nov 13 13:52:51 missy kernel: vlan1: changing name to 'ix3.2'
Nov 13 13:52:51 missy check_reload_status: Restarting ipsec tunnels
Nov 13 13:53:22 missy missy.lab.jimp.pw nginx: 2018/11/13 13:53:22 [error] 16591#100204: *1 upstream timed out (60: Operation timed out) while reading response header from upstream, client: 198.51.100.108, server: , request: "POST /interfaces.php?if=opt5 HTTP/2.0", upstream: "fastcgi://unix:/var/run/php-fpm.socket", host: "missy.lab.jimp.pw", referrer: "https://missy.lab.jimp.pw/interfaces.php?if=opt5" 
Nov 13 13:54:05 missy check_reload_status: updating dyndns opt6
Nov 13 13:54:05 missy check_reload_status: Restarting ipsec tunnels
Nov 13 13:54:05 missy kernel: vlan2: changing name to 'ix3.3'
Nov 13 13:55:43 missy check_reload_status: updating dyndns opt7
Nov 13 13:55:44 missy kernel: vlan3: changing name to 'ix3.4'
Nov 13 13:55:44 missy check_reload_status: Restarting ipsec tunnels
Nov 13 13:57:41 missy check_reload_status: updating dyndns opt8
Nov 13 13:57:41 missy check_reload_status: Restarting ipsec tunnels
Nov 13 13:57:41 missy kernel: vlan4: changing name to 'ix3.5'
Nov 13 13:59:39 missy check_reload_status: updating dyndns opt9
Nov 13 13:59:40 missy kernel: vlan5: changing name to 'ix3.6'
Nov 13 13:59:40 missy check_reload_status: Restarting ipsec tunnels
Nov 13 14:01:36 missy check_reload_status: updating dyndns opt10
Nov 13 14:01:37 missy kernel: vlan6: changing name to 'ix3.7'
Nov 13 14:01:37 missy check_reload_status: Restarting ipsec tunnels
Nov 13 14:03:09 missy check_reload_status: updating dyndns opt11
Nov 13 14:03:09 missy check_reload_status: Restarting ipsec tunnels
Nov 13 14:03:09 missy kernel: vlan7: changing name to 'ix3.8'

The contents of /tmp/.interfaces.apply also agree that only one interface was touched:

a:1:{s:4:"opt5";a:2:{s:5:"ifcfg";a:7:{s:5:"descr";s:3:"VL1";s:2:"if";s:5:"ix3.1";s:6:"enable";s:0:"";s:6:"ipaddr";s:10:"198.18.1.1";s:6:"subnet";s:2:"24";s:8:"spoofmac";s:0:"";s:6:"realif";s:5:"ix3.1";}s:4:"ppps";a:0:{}}}

Occasionally a PHP thread will go up to 100% CPU most most of the time is spent idle/waiting.

This is on an SG-5100, but the customer in the thread above saw it on an XG-1541.

Associated revisions

Revision 433a8e71 (diff)
Added by Luiz Souza 6 months ago

Do not call interfaces_vlan_configure() every time an interface is edited in GUI.

This is just necessary when a parent interface is changed and we have to propagate the changes to all clones (MTU, FLAGS and others).

Add a logic to detect when a parent interface is changed and only then call interfaces_vlan_configure().

Ticket #9115

Revision c9f69485 (diff)
Added by Luiz Souza 6 months ago

Do not call interfaces_vlan_configure() every time an interface is edited in GUI.

This is just necessary when a parent interface is changed and we have to propagate the changes to all clones (MTU, FLAGS and others).

Add a logic to detect when a parent interface is changed and only then call interfaces_vlan_configure().

Ticket #9115

(cherry picked from commit 433a8e71f3b68c39634e11b62d8bf3d9e8ec878c)

History

#1 Updated by Jim Pingle 6 months ago

  • Subject changed from A large number of VLANs causes PHP to spin at 100% when making an interface change to A large number of VLANs causes PHP issues when making an interface change

#2 Updated by Renato Botelho 6 months ago

  • Assignee set to Luiz Souza
  • Target version changed from 48 to 2.4.4-p1

#3 Updated by Renato Botelho 6 months ago

  • Status changed from New to Feedback

#4 Updated by Luiz Souza 6 months ago

  • % Done changed from 0 to 100

This regression is now fixed and only when really necessary the VLANs will be recreated.

#5 Updated by Jim Pingle 6 months ago

Looks a lot better here with the new method. Editing the parent is still a problem, however, but that can be split off into its own issue.

#6 Updated by Jim Pingle 6 months ago

  • Status changed from Feedback to Resolved

I split the parent interface issue off to #9154 -- this one can be closed.

Also available in: Atom PDF