Project

General

Profile

Actions
Copy link

Feature #5133

closed

syslog repeated message suppression

Added by Brian Candler almost 10 years ago. Updated almost 10 years ago.

Status:
Rejected
Priority:
Normal
Assignee:
-
Category:
-
Target version:
-
Start date:
09/14/2015
Due date:
% Done:

0%

Estimated time:
Plus Target Version:
Release Notes:

Description

Could pfsense gain the ability to suppress/summarise repeated identical syslog messages?

The box shown here is 2.1-RELEASE (unfortunately too critical to upgrade right now)

After installing the snort package, syslog started sending approximately 153 messages per second of the following form:

2015 Sep 14 08:59:59 [Select()]: Failed to execute  query [SELECT vseq FROM `schema`] , will retry
2015 Sep 14 08:59:59 [Select()]: Failed to execute  query [SELECT vseq FROM `schema`] , will retry
2015 Sep 14 08:59:59 [Select()]: Failed to execute  query [SELECT vseq FROM `schema`] , will retry
2015 Sep 14 08:59:59 [Select()]: Failed to execute  query [SELECT vseq FROM `schema`] , will retry
2015 Sep 14 08:59:59 [Select()]: Failed to execute  query [SELECT vseq FROM `schema`] , will retry
2015 Sep 14 08:59:59 [Select()]: Failed to execute  query [SELECT vseq FROM `schema`] , will retry
2015 Sep 14 08:59:59 [Select()]: Failed to execute  query [SELECT vseq FROM `schema`] , will retry
2015 Sep 14 08:59:59 [Select()]: Failed to execute  query [SELECT vseq FROM `schema`] , will retry
2015 Sep 14 08:59:59 [Select()]: Failed to execute  query [SELECT vseq FROM `schema`] , will retry
2015 Sep 14 08:59:59 [Select()]: Failed to execute  query [SELECT vseq FROM `schema`] , will retry

Load average 0.63, top shows:

  PID USERNAME  THR PRI NICE   SIZE    RES STATE   C   TIME   WCPU COMMAND
56734 root        1  59    0 49736K 31304K CPU1    1  37.1H 24.76% barnyard2
67670 root        1  58    0  6956K  1620K select  3  28.4H 21.68% syslogd

This is apparently an issue with barnyard2 database setup, which may or may not be fixed in a newer version of pfsense.

However, the way we found this was when our central syslog-ng server started growing its logs at ~30GB per day, and we got alerted when disks were nearly full.

syslog-ng doesn't appear to do repeated message suppression, hence the interest in whether it could be configured at pfsense side to protect against similar issues.

Actions
Copy link
 #1

Updated by Brian Candler almost 10 years ago

BTW: for anyone who reaches this issue because of the barnyard2 problem: pfsense had been configured with option "Enable logging of alerts to a MySQL database instance" and had been given a valid username/password/database and grants, but no tables had been created in the database.

The following script creates the necessary tables:
https://raw.githubusercontent.com/eldondev/Snort/master/schemas/create_mysql

Actions
Copy link
 #2

Updated by Kill Bill almost 10 years ago

Hmmm... Considering the VRT rules for 2.1.x Snort version are gone, I think this issue is not exactly urgent for you any more. :P

Actions
Copy link
 #3

Updated by Jim Pingle almost 10 years ago

  • Project changed from pfSense Packages to pfSense
  • Status changed from New to Rejected

We purposefully disable duplicate message compression in all cases explicitly by passing "-c -c" to syslogd. Someone could add a toggle option to the log settings to handle that, but we don't recommend it. That would be a feature request, not a bug, and not in packages.

Your issue with snort is separate from this issue, however.

Actions
Copy link

Also available in: Atom PDF