pfSense

Could pfsense gain the ability to suppress/summarise repeated identical syslog messages?

The box shown here is 2.1-RELEASE (unfortunately too critical to upgrade right now)

After installing the snort package, syslog started sending approximately 153 messages per second of the following form:

2015 Sep 14 08:59:59 [Select()]: Failed to execute  query [SELECT vseq FROM `schema`] , will retry
2015 Sep 14 08:59:59 [Select()]: Failed to execute  query [SELECT vseq FROM `schema`] , will retry
2015 Sep 14 08:59:59 [Select()]: Failed to execute  query [SELECT vseq FROM `schema`] , will retry
2015 Sep 14 08:59:59 [Select()]: Failed to execute  query [SELECT vseq FROM `schema`] , will retry
2015 Sep 14 08:59:59 [Select()]: Failed to execute  query [SELECT vseq FROM `schema`] , will retry
2015 Sep 14 08:59:59 [Select()]: Failed to execute  query [SELECT vseq FROM `schema`] , will retry
2015 Sep 14 08:59:59 [Select()]: Failed to execute  query [SELECT vseq FROM `schema`] , will retry
2015 Sep 14 08:59:59 [Select()]: Failed to execute  query [SELECT vseq FROM `schema`] , will retry
2015 Sep 14 08:59:59 [Select()]: Failed to execute  query [SELECT vseq FROM `schema`] , will retry
2015 Sep 14 08:59:59 [Select()]: Failed to execute  query [SELECT vseq FROM `schema`] , will retry

Load average 0.63, top shows:

  PID USERNAME  THR PRI NICE   SIZE    RES STATE   C   TIME   WCPU COMMAND
56734 root        1  59    0 49736K 31304K CPU1    1  37.1H 24.76% barnyard2
67670 root        1  58    0  6956K  1620K select  3  28.4H 21.68% syslogd

This is apparently an issue with barnyard2 database setup, which may or may not be fixed in a newer version of pfsense.

• <http://seclists.org/snort/2014/q3/865>
• <http://seclists.org/snort/2014/q3/882>
• <http://eth0.us/node/240>

However, the way we found this was when our central syslog-ng server started growing its logs at ~30GB per day, and we got alerted when disks were nearly full.

syslog-ng doesn't appear to do repeated message suppression, hence the interest in whether it could be configured at pfsense side to protect against similar issues.