pfSense

Chris Buechler wrote:

that's 2.3(.0)_1 rather than 2.3.1. It wasn't 2.3->2.3_1 that did it, since that only upgraded ntpd, rather something that would have happened on 2.3 as well. I'm guessing it's one of two things. Either something related to #6177, or there seems to be some kind of issue with the IPsec dashboard widget causing that to happen for a few people.

If this is replicable for you, if you have the IPsec dashboard widget enabled, please try to remove that and see if that fixes the problem. That'll at least tell us where the issue resides.

I have seen this issue frequently on clients since 2.3 rolled. I was more concerned with #6296 which was causing me many headaches, but I will try removing the IPSec widget on a few sites and report back (I have one with the webgui locked up right now who is a prime candidate). Any log dumps that would be helpful?

Restarting the webconfigurator from the console does not resolve the issue.
Other than the web not functioning, the firewall is performing as normal.

Try restarting PHP-FPM from the console. That seems to clear up the issue for me...

Brent Kerlin wrote:

I have seen this issue frequently on clients since 2.3 rolled. I was more concerned with #6296 which was causing me many headaches, but I will try removing the IPSec widget on a few sites and report back (I have one with the webgui locked up right now who is a prime candidate). Any log dumps that would be helpful?

I have removed the IPSec Widget from all the sites at which I was having this PHP-FPM issue. I'll report back in a couple days or if the problem persists.

I have no issues since removing the IPsec widget. Now on 2.3.1 and have not seen a lockup

I have looked through the code again and nothing really stands out.

It would be helpful to know:

• How many tunnels do people have in cases where the issue is seen?
• Does it make any difference if the widget is set to show Overview, Tunnels, or Mobile?

THanks!

Steve Beaver wrote:

I have looked through the code again and nothing really stands out.

Ditto. Heard of roughly a handful of reports of this, but never seen it myself. Additional details would be appreciated.

Thanks to Alex for getting me into an affected system. It's occasionally getting stuck in pfSense_ipsec_list_sa, without triggering any of the printfs there.

PHP_FUNCTION(pfSense_ipsec_list_sa) {

        vici_conn_t *conn;
        vici_req_t *req;
        vici_res_t *res;

        array_init(return_value);

        vici_init();
        conn = vici_connect(NULL);
        if (conn) {
                if (vici_register(conn, "list-sa", build_ipsec_sa_array, (void *) return_value) != 0) {
                        php_printf("VICI registration failed: %s\n", strerror(errno));
                } else {
                        req = vici_begin("list-sas");
                        res = vici_submit(req, conn);
                        if (res) {
                                vici_free_res(res);
                        }
                }
                vici_disconnect(conn);
        } else {
                php_printf("VICI connection failed: %s\n", strerror(errno));
        }

        vici_deinit();

}

What I committed on this ticket should prevent this (and many other possible failure scenarios with commands that don't return) from killing the GUI. request_terminate_timeout will kill them off after 900 seconds. It only happens once every few minutes when continually refreshing a page that uses that function, so that's been enough to keep Alex's system from killing the GUI again.

We are also having what appears to be the same issue running on version 2.3.2 As a side affect of php-fpm going down our OpenVPN clients also lose the ability to connect until we restart php-fpm and openvpn. I don't know if this would affect all OpenVPN or just those using ssl cert authentication as is the case with our setup.

Nick Wenos wrote:

We are also having what appears to be the same issue running on version 2.3.2 As a side affect of php-fpm going down our OpenVPN clients also lose the ability to connect until we restart php-fpm and openvpn. I don't know if this would affect all OpenVPN or just those using ssl cert authentication as is the case with our setup.

We are also seeing this on 2.3.3
Running netstat -an shows request filling up the Recv-Q for IPC connection to /var/run/php-fpm.socket.

I am also seeing this on 2.3.3. Is there any known work around other than removing the ipsec widget? Maybe changing the polling frequency?

I think this bug's priority should be raised since it also breaks openvpn functionality.

Sounds like a fantastic change. Thanks Steve