pfSense

This typically happens when you have a corrupted log file. The first attempted write to said log file will crash syslogd. Reset the log files from Status > System Logs, Settings tab.

That said, we probably need to come up with a way to detect broken/corrupted log files, assuming your problem is from that same cause.

It looks like this fixed my issue. I had the log size set to 8192 thinking that it was in KB and realized that the field is in bytes so I was setting it to 8K when the default shows 500KB. Im guessing that it was corrupting because the log file was to small and unable to rollover right after boot is complete.

I just had this happen again after rebooting my box from Bug #4392. How can I email you the logs?

If you have made an archive of the logs you can post them on a site like Google Drive or Dropbox and PM me the URL on the forum or send it to me directly, though it may be too large to e-mail. Address is jimp (at) pfsense [dot] org.

I sent you an email yesterday with the logs (9MB). Did you get the email?

I received it, but it may be Tuesday before I have a chance to look at the logs.

I still can't seem to reproduce it here even with a "bad" set of logs and similar settings. There must be some other contributing factor at work.

I'm curious if lighttpd will produce any errors if/when syslogd breaks when saving the syslog settings. To enable some extra logging in lighttpd, apply this patch:

http://files.atx.pfsense.org/jimp/patches/lighttpd-breakage-log.diff (don't apply that now, see Chris' later message)

And then from the console, use option 11 to restart lighttpd. Try clicking Save on Status > System Logs, Settings tab and then if syslogd is not running, check /var/log/lighttpd-breakage.log to see if anything shows up there.

I applied the patch and it created the lighttpd-breakage.log but its 0bytes. After applying the patch and restarting ligttpd here is what was logged in system log.

Feb 18 08:34:35 syslogd: kernel boot file is /boot/kernel/kernel
Feb 18 08:34:35 syslogd: exiting on signal 15
Feb 18 08:34:35 check_reload_status: Syncing firewall
Feb 18 08:33:42 php: rc.restart_webgui: Creating rrd update script
Feb 18 08:33:29 sshd¹²⁴⁴⁹: Accepted keyboard-interactive/pam for admin from xx:xx:xx:xx port 55839 ssh2
Feb 18 08:32:18 check_reload_status: Syncing firewall
Feb 18 08:31:20 check_reload_status: Syncing firewall
Feb 18 08:29:48 check_reload_status: Syncing firewall
Feb 18 08:28:34 check_reload_status: Reloading filter
Feb 18 08:28:34 php-fpm¹⁸⁷³²: /pkg_mgr_install.php: Successfully installed package: System Patches.
Feb 18 08:28:34 check_reload_status: Syncing firewall
Feb 18 08:28:34 check_reload_status: Syncing firewall
Feb 18 08:28:32 check_reload_status: Syncing firewall
Feb 18 08:28:31 php-fpm¹⁸⁷³²: /pkg_mgr_install.php: Beginning package installation for System Patches .

I havent seen this bug crop up on my other 2 systems so I'm beginning to wonder if its just something very specific to this machine. My other 2 systems were on 2.1.5 had inplace upgrades to 2.2-RELEASE. The system that is having this issue was a new install on 2.2RC (not sure what daily snapshot) and upgraded through the daily snapshots to 2.2-RELEASE. I thinking I should try rebuilding this machine from the 2.2-RELEASE and see if that solves the issue. All 3 systems have the same base hardware, however this box also has a Broadcom accelerator card (cryptodev) and and Atheros AR9380 wireless card (AP mode).

No need for that patch I posted now, Chris found the actual fix after that. The fix is in the commits shown in the "Associated revisions" section here on the ticket.

I have applied the patch. Ill let you know if I see any more syslog crashes. Thanks!

Albert Yang wrote:

Im not sure if it was fixed for 2.2.4? But seems to effect it also?

It does not, this has been fixed since 2.2.1. Your log clearly shows it's starting again. Post to the forum with more info on what you're trying to do.

Hi Chris Thank you for the reply when you say it clearly shows its starting again but why the syslogd: exiting on signal 15 im running ELK which pfSense sends all the logs and for some reason wont work i posted on the forums but no luck so far :(

https://forum.pfsense.org/index.php?topic=110234.msg613717#msg613717

Thank you