Project

General

Profile

Bug #6235

Snort sometimes crashes during rule update process (specifically related to VRT .so rule update?)

Added by Andrew W over 3 years ago. Updated about 1 year ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
Snort
Target version:
-
Start date:
04/22/2016
Due date:
% Done:

0%

Estimated time:
Affected Version:
Affected Architecture:

Description

Snort occasionally crashes during the rule update process and doesn't start again until I manually restart it via the GUI. When the crash occurs, it usually crashes with signal 11 (SIGSEGV), but most recently it crashed with signal 4 (SIGILL). The logs from the latest crash are as follows:

Apr 20 04:00:01 x php: /usr/local/pkg/snort/snort_check_for_rule_updates.php: [Snort] There is a new set of Snort VRT rules posted. Downloading snortrules-snapshot-2980.tar.gz...
Apr 20 04:00:13 x php: /usr/local/pkg/snort/snort_check_for_rule_updates.php: [Snort] Snort VRT rules file update downloaded successfully
Apr 20 04:00:13 x php: /usr/local/pkg/snort/snort_check_for_rule_updates.php: [Snort] Snort OpenAppID detectors are up to date...
Apr 20 04:00:14 x php: /usr/local/pkg/snort/snort_check_for_rule_updates.php: [Snort] There is a new set of Emerging Threats Open rules posted. Downloading emerging.rules.tar.gz...
Apr 20 04:00:15 x php: /usr/local/pkg/snort/snort_check_for_rule_updates.php: [Snort] Emerging Threats Open rules file update downloaded successfully
Apr 20 04:00:26 x php: /usr/local/pkg/snort/snort_check_for_rule_updates.php: [Snort] Updating rules configuration for: LAN ...
Apr 20 04:00:28 x kernel: pid 65324 (snort), uid 0: exited on signal 4
Apr 20 04:00:28 x kernel: igb1: promiscuous mode disabled
Apr 20 04:00:42 x php: /usr/local/pkg/snort/snort_check_for_rule_updates.php: [Snort] Enabling any flowbit-required rules for: LAN...
Apr 20 04:00:44 x php: /usr/local/pkg/snort/snort_check_for_rule_updates.php: [Snort] Building new sid-msg.map file for LAN...
Apr 20 04:00:51 x php: /usr/local/pkg/snort/snort_check_for_rule_updates.php: [Snort] The Rules update has finished.
Apr 20 04:00:51 x check_reload_status: Syncing firewall
[End of logs for Apr 20]

From /var/log/snort/snort_rules_update.log:

Starting rules update...  Time: 2016-04-20 04:00:00
        Downloading Snort VRT rules md5 file snortrules-snapshot-2980.tar.gz.md5...
        Checking Snort VRT rules md5 file...
        There is a new set of Snort VRT rules posted.
        Downloading file 'snortrules-snapshot-2980.tar.gz'...
        Done downloading rules file.
        Downloading Snort OpenAppID detectors md5 file snort-openappid.tar.gz.md5...
        Checking Snort OpenAppID detectors md5 file...
        Snort OpenAppID detectors are up to date.
        Downloading Emerging Threats Open rules md5 file emerging.rules.tar.gz.md5...
        Checking Emerging Threats Open rules md5 file...
        There is a new set of Emerging Threats Open rules posted.
        Downloading file 'emerging.rules.tar.gz'...
        Done downloading rules file.
        Extracting and installing Snort VRT rules...
        Using Snort VRT precompiled SO rules for FreeBSD-10-0 ...
        Installation of Snort VRT rules completed.
        Extracting and installing Emerging Threats Open rules...
        Installation of Emerging Threats Open rules completed.
        Copying new config and map files...
        Updating rules configuration for: LAN ...
The Rules update has finished.  Time: 2016-04-20 04:00:51

Notice above that the Snort VRT precompiled SO rules were updated - on previous days that these rules were not updated, the snort update process completed successfully without the snort process crashing. I wonder if the .so's are being modified such that the running snort process will crash upon trying to call a function provided by the updated libraries (after the function address has been resolved based on the old .so).

PfSense Machine Specs:

Netgate RCC-DFF    
Intel(R) Atom(TM) CPU C2338 @ 1.74GHz
2.3-RELEASE (amd64) 
built on Mon Apr 11 18:28:29 CDT 2016 
FreeBSD 10.3-RELEASE

I've had these problems for the last few months (running 2.2.6 as well.)

Please let me know what additional information I can provide. Thanks!

History

#1 Updated by Chris Buechler over 3 years ago

  • Project changed from pfSense to pfSense Packages
  • Category set to Snort

#2 Updated by Bill Meeks over 3 years ago

The rules update process will restart any running Snort processes at the end of the update. However, I guess it is possible that the currently running Snort process might try to access the shared object (*.so) rules during the instant of update and have a problem with that. Purely conjecture, though, on my part.

I have not noticed this problem on my personal system, but it is a home firewall and does not see much traffic.

This one might be hard to nail down since it is intermittent. Keep an eye on it to see if it happens again and if the crash corresponds with a fresh set of VRT rules. Those shared object rules will get updated on each Snort VRT rules package update. Those generally come out on Tuesdays and Thursdays US Eastern Time.

Bill

#3 Updated by Andrew W over 3 years ago

This happens fairly frequently for me - it crashed again on Apr. 22nd when pulling in the latest VRT rules (SIGSEGV).

If I can make snort generate a core dump file the next time it crashes, that might point to whether it is crashing when attempting to call a function in one of the .so's. I haven't figured out exactly how to make snort generate a coredump when it crashes, though. From reading online, hopefully the following will work?

sysctl kern.corefile=/tmp/%N.core
sysctl kern.capmode_coredump=1
sysctl kern.sugid_coredump=1

#4 Updated by Frank Seidel about 1 year ago

I also have the same problem.. nearly every night on the rule update process the snort service
dies or isn't coming up again after the updates ran.
(running pfSense community edition 2.4.3-RELEASE-p1 (amd64) with snort 3.2.9.6_1)

Jul 20 02:06:50 check_reload_status Syncing firewall
Jul 20 02:06:49 php /usr/local/pkg/snort/snort_check_for_rule_updates.php: [Snort] The Rules update has finished.
Jul 20 02:06:42 php /usr/local/pkg/snort/snort_check_for_rule_updates.php: [Snort] Building new sid-msg.map file for LAN...
Jul 20 02:06:39 php /usr/local/pkg/snort/snort_check_for_rule_updates.php: [Snort] Enabling any flowbit-required rules for: LAN...
Jul 20 02:06:15 kernel igb2: promiscuous mode disabled
Jul 20 02:06:15 kernel pid 2526 (snort), uid 0: exited on signal 4
Jul 20 02:06:09 php /usr/local/pkg/snort/snort_check_for_rule_updates.php: [Snort] Updating rules configuration for: LAN ...
Jul 20 02:05:33 php /usr/local/pkg/snort/snort_check_for_rule_updates.php: [Snort] Emerging Threats Open rules file update downloaded successfully
Jul 20 02:05:29 php /usr/local/pkg/snort/snort_check_for_rule_updates.php: [Snort] There is a new set of Emerging Threats Open rules posted. Downloading emerging.rules.tar.gz...
Jul 20 02:05:28 php /usr/local/pkg/snort/snort_check_for_rule_updates.php: [Snort] Snort OpenAppID RULES detectors file update downloaded successfully
Jul 20 02:05:27 php /usr/local/pkg/snort/snort_check_for_rule_updates.php: [Snort] There is a new set of Snort OpenAppID RULES detectors posted. Downloading appid_rules.tar.gz...
Jul 20 02:05:27 php /usr/local/pkg/snort/snort_check_for_rule_updates.php: [Snort] Snort OpenAppID detectors are up to date...
Jul 20 02:05:25 php /usr/local/pkg/snort/snort_check_for_rule_updates.php: [Snort] Snort Subscriber rules file update downloaded successfully
Jul 20 02:05:05 php /usr/local/pkg/snort/snort_check_for_rule_updates.php: [Snort] There is a new set of Snort Subscriber rules posted. Downloading snortrules-snapshot-29111.tar.gz...

Also available in: Atom PDF