Project

General

Profile

Bug #6813

2.3.3 built on Fri Sep 23 11:34:50 CDT 2016 - segfaulting processes result in non-functional system

Added by Kill Bill almost 3 years ago. Updated about 2 years ago.

Status:
Resolved
Priority:
Urgent
Category:
Operating System
Target version:
Start date:
09/24/2016
Due date:
% Done:

100%

Estimated time:
Affected Version:
Affected Architecture:
All

Description

https://forum.pfsense.org/index.php?topic=118714.0

No default routes configured, I can SSH in remotely only via IPv6, even after adding those missing default routes manually I cannot access pretty much anything behind the firewall. Kindly provide a way to downgrade to some previous working snapshot ASAP. :-(

History

#1 Updated by Kill Bill almost 3 years ago

Err, "only via IPv4". Whatever.

#2 Updated by Kill Bill almost 3 years ago

Also, OpenVPN is completely no-op, producing just segfaults.

https://forum.pfsense.org/index.php?topic=118709.0

#3 Updated by Jim Pingle almost 3 years ago

  • Subject changed from 2.3.3 built on Fri Sep 23 11:34:50 CDT 2016 - totally broken networking to 2.3.3 built on Fri Sep 23 11:34:50 CDT 2016 - segfaulting processes result in non-functional system
  • Status changed from New to Confirmed

Setup a new VM as a clean test. Works on a snap from the 22nd, same one updated to the 23rd breaks in various ways. The network layer is fine but PHP and other things are segfaulting. It would appear that on systems with non-functioning networks, PHP is crashing before or during the network setup, which is why the symptoms vary.

Sep 24 11:56:07 pfSense kernel: pid 359 (php-cgi), uid 0: exited on signal 11 (core dumped)

#4 Updated by Greg M almost 3 years ago

Pull it :|

#5 Updated by Kill Bill almost 3 years ago

All the crashes here seem to be OpenVPN related.

Calling openvpn_resync_all() causes php-cgi to crash on boot, which unfortunately screws the rest of network setup.

Creating wireless clone interfaces...done.
Configuring LAGG interfaces...done.
Configuring VLAN interfaces...done.
Configuring QinQ interfaces...done.
Configuring WAN interface...done.
Configuring LAN interface...done.
Configuring MGMT interface...done.
Configuring WLAN interface...done.
Configuring HEIPV6 interface...done.
Configuring CARP settings...done.
Syncing OpenVPN settings...Segmentation fault (core dumped)
Starting CRON... done.

Going to VPN - OpenVPN menu causes php-fpm to segfault.

Sep 24 21:12:15 gw gw.example.com nginx: 2016/09/24 21:12:15 [error] 24339#100138: *6627 upstream prematurely closed connection while reading response header from upstream, client: 192.0.2.3, server: , request: "GET /vpn_openvpn_server.php?act=edit&id=0 HTTP/1.1", upstream: "fastcgi://unix:/var/run/php-fpm.socket:", host: "gw.example.com" 
Sep 24 21:12:15 gw kernel: pid 19542 (php-fpm), uid 0: exited on signal 11 (core dumped)

#6 Updated by Renato Botelho almost 3 years ago

  • Status changed from Confirmed to Feedback
  • Assignee set to Renato Botelho
  • Target version set to 2.3.2-p1
  • % Done changed from 0 to 100

A fix was committed in FreeBSD and now imported to our repo. Next round of snapshots should be fixed

https://svnweb.freebsd.org/base?view=revision&revision=306336

#7 Updated by Kill Bill almost 3 years ago

Fixed, thanks.

#8 Updated by Renato Botelho almost 3 years ago

  • Status changed from Feedback to Resolved

#9 Updated by Kill Bill almost 3 years ago

As a follow up - it appears that after openvpn_resync_all(), nothing else below that function call could run from the /etc/rc.bootup script -- since almost everything is plopped into that one script that crashed php-cgi.

Shouldn't this thing be a whole lot more robust/failsafe? As in, e.g. not sticking almost the entire boot process into one script which - when crashed somewhere at the top of the calls stack, renders the system very much useless? So, the core of the issue doesn't seem to be resolved at all, can happen anytime again, with amount of breakage depending on how early in the script it happens.

Do you want a separate bug for that?

#10 Updated by Greg Toombs about 2 years ago

Kill Bill wrote:

Going to VPN - OpenVPN menu causes php-fpm to segfault.

I'm not using OpenVPN, nor accessing the menu, but I still get a segfault that looks similar to what you showed. I've filed #7819

#11 Updated by Kill Bill about 2 years ago

Greg Toombs wrote:

I'm not using OpenVPN, nor accessing the menu, but I still get a segfault that looks similar to what you showed. I've filed #7819

Yeah this old bug should be left well alone, completely different code base.

Also available in: Atom PDF