Feature #5133
closedsyslog repeated message suppression
0%
Description
Could pfsense gain the ability to suppress/summarise repeated identical syslog messages?
The box shown here is 2.1-RELEASE (unfortunately too critical to upgrade right now)
After installing the snort package, syslog started sending approximately 153 messages per second of the following form:
2015 Sep 14 08:59:59 [Select()]: Failed to execute query [SELECT vseq FROM `schema`] , will retry 2015 Sep 14 08:59:59 [Select()]: Failed to execute query [SELECT vseq FROM `schema`] , will retry 2015 Sep 14 08:59:59 [Select()]: Failed to execute query [SELECT vseq FROM `schema`] , will retry 2015 Sep 14 08:59:59 [Select()]: Failed to execute query [SELECT vseq FROM `schema`] , will retry 2015 Sep 14 08:59:59 [Select()]: Failed to execute query [SELECT vseq FROM `schema`] , will retry 2015 Sep 14 08:59:59 [Select()]: Failed to execute query [SELECT vseq FROM `schema`] , will retry 2015 Sep 14 08:59:59 [Select()]: Failed to execute query [SELECT vseq FROM `schema`] , will retry 2015 Sep 14 08:59:59 [Select()]: Failed to execute query [SELECT vseq FROM `schema`] , will retry 2015 Sep 14 08:59:59 [Select()]: Failed to execute query [SELECT vseq FROM `schema`] , will retry 2015 Sep 14 08:59:59 [Select()]: Failed to execute query [SELECT vseq FROM `schema`] , will retry
Load average 0.63, top shows:
PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND 56734 root 1 59 0 49736K 31304K CPU1 1 37.1H 24.76% barnyard2 67670 root 1 58 0 6956K 1620K select 3 28.4H 21.68% syslogd
This is apparently an issue with barnyard2 database setup, which may or may not be fixed in a newer version of pfsense.
- <http://seclists.org/snort/2014/q3/865>
- <http://seclists.org/snort/2014/q3/882>
- <http://eth0.us/node/240>
However, the way we found this was when our central syslog-ng server started growing its logs at ~30GB per day, and we got alerted when disks were nearly full.
syslog-ng doesn't appear to do repeated message suppression, hence the interest in whether it could be configured at pfsense side to protect against similar issues.
Updated by Brian Candler over 8 years ago
BTW: for anyone who reaches this issue because of the barnyard2 problem: pfsense had been configured with option "Enable logging of alerts to a MySQL database instance" and had been given a valid username/password/database and grants, but no tables had been created in the database.
The following script creates the necessary tables:
https://raw.githubusercontent.com/eldondev/Snort/master/schemas/create_mysql
Updated by Kill Bill over 8 years ago
Hmmm... Considering the VRT rules for 2.1.x Snort version are gone, I think this issue is not exactly urgent for you any more. :P
Updated by Jim Pingle over 8 years ago
- Project changed from pfSense Packages to pfSense
- Status changed from New to Rejected
We purposefully disable duplicate message compression in all cases explicitly by passing "-c -c" to syslogd. Someone could add a toggle option to the log settings to handle that, but we don't recommend it. That would be a feature request, not a bug, and not in packages.
Your issue with snort is separate from this issue, however.