Project

General

Profile

Actions

Bug #7278

closed

Suricata Service - Advanced Configuration Pass-Through not working

Added by Michael Strasner about 7 years ago. Updated over 6 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
-
Category:
Suricata
Target version:
-
Start date:
02/18/2017
Due date:
% Done:

0%

Estimated time:
Plus Target Version:
Affected Version:
2.3.2
Affected Plus Version:
Affected Architecture:

Description

  • Issue: Advanced Configuration Pass-Through not working under pfSense > Services > Suricata > Edit Interface Settings - WAN (I'm using the WAN interface)

Pfsense Version: 2.3.2-Release
Suricata Version: 3.1.2_2

  • Reproduction:
  • Add the Suricata Service
  • Edit either of the two .yaml files available in the shell (as root)
  • find / -name '*.yaml'

/usr/local/etc/suricata/suricata.yaml
/usr/local/etc/suricata/suricata_20934_ix1/suricata.yaml

  • Edit with vi, save.
  • Reload Suricata
    Suricata reloads, and rebuilds configuration files from Pfsense options (notice the time stamps):

rwxr-xr-x 3 root wheel 512 Feb 18 02:04 .
drwxr-xr-x 4 root wheel 512 Feb 18 02:04 ..
rw-r--r- 1 root wheel 2888 Feb 18 16:49 classification.config
rw-r--r- 1 root wheel 185 Feb 18 16:49 passlist
rw-r--r- 1 root wheel 1332 Feb 18 16:49 reference.config
drwxr-xr-x 2 root wheel 512 Feb 18 02:04 rules
rw-r--r- 1 root wheel 2485735 Feb 18 16:49 sid-msg.map
rw-r--r- 1 root wheel 8927 Feb 18 16:49 suricata.yaml
rw-r--r- 1 root wheel 0 Feb 18 16:49 threshold.config
rw-r--r- 1 root wheel 53841 Feb 18 16:49 unicode.map

drwxr-xr-x 3 root wheel 512 Feb 18 02:04 .
drwxr-xr-x 4 root wheel 512 Feb 18 02:04 ..
rw-r--r- 1 root wheel 2888 Feb 18 17:10 classification.config
rw-r--r- 1 root wheel 185 Feb 18 17:10 passlist
rw-r--r- 1 root wheel 1332 Feb 18 17:10 reference.config
drwxr-xr-x 2 root wheel 512 Feb 18 02:04 rules
rw-r--r- 1 root wheel 2485735 Feb 18 17:10 sid-msg.map
rw-r--r- 1 root wheel 8927 Feb 18 17:10 suricata.yaml
rw-r--r- 1 root wheel 0 Feb 18 17:10 threshold.config
rw-r--r- 1 root wheel 53841 Feb 18 17:10 unicode.map

  • Check the loaded configuration: ps auxwww | grep suricata

root 52501 0.1 1.3 561304 418060 - Ss 5:10PM 0:11.72 /usr/local/bin/suricata -i ix1 -D -c /usr/local/etc/suricata/suricata_20934_ix1/suricata.yaml --pidfile /var/run/suricata_ix120934.pid

Contents of Advanced Configuration Pass-Through not parsed into the new suricata.yaml configuration file, after reload

  • Add the configuration to Services > Suricata > Edit Interface Settings - WAN (I'm using the WAN interface)> Advanced Configuration Pass- Through
  • Recheck the /usr/local/etc/suricata/suricata_20934_ix1/suricata.yaml file
    The added configuration does not load the Advanced Configuration Pass-Through contents (this what I have in Advanced Configuration Pass - Through):

threading:
set-cpu-affinity: yes
- management-cpu-set:
cpu: [ 0 ] # include only these cpus in affinity settings
- receive-cpu-set:
cpu: [ 1 ] # include only these cpus in affinity settings
- decode-cpu-set:
cpu: [ "2" ]
mode: "balanced"
- stream-cpu-set:
cpu: [ "0-3" ]
- detect-cpu-set:
cpu: [ "4,6" ]
mode: "exclusive" # run detect threads in these cpus # Use explicitely 3 threads and don't compute number by using # detect-thread-ratio variable:
threads: 3
prio:
low: [ "0-3" ]
medium: [ "5-7" ]
default: "medium"
- verdict-cpu-set:
cpu: [ 0 ]
prio:
default: "high"
- reject-cpu-set:
cpu: [ 0 ]
prio:
default: "low"
- output-cpu-set:
cpu: [ "0" ]
prio:
default: "medium"

detect:
profile: custom
custom-values:
toclient-groups: 200
toserver-groups: 200
sgh-mpm-context: auto
inspection-recursion-limit: 3000

Notice the suricata.yml file actual contents attached (does not include the added configuration in Advanced Configuration Pass-Through

The first tune for cpu-affinity (threading) found here: https://home.regit.org/2011/01/optimizing-suricata-on-a-multicore-cpu/
The second tune for Memory found here: http://suricata.readthedocs.io/en/latest/performance/high-performance-config.html

  • Hardware:
    I have low power server Xeon, with high memory seeking to tune Suricata (set and then forget, basically)

hw.model: Intel(R) Xeon(R) CPU D-1518 @ 2.20GHz
hw.machine: amd64
hw.ncpu: 8
real memory = 34359738368 (32768 MB)
avail memory = 33147830272 (31612 MB)

  • Result:
    Pfsense is not parsing the
    Advanced Configuration Pass-Through.
  • Affected: Unable to tune advanced features in the Suricata configuration for Branch/Office Hardware

Files

suricata.yaml (8.72 KB) suricata.yaml Suricata YML does not receive Advanced Configuration Pass-Through data Michael Strasner, 02/18/2017 04:32 PM
Actions #1

Updated by Kill Bill about 7 years ago

Please, use the pre button to post code/command output. This is just unreadable mess.

Actions #2

Updated by Michael Strasner about 7 years ago

This is just ... mess.

Interesting wording, that's what I thought of the feature.

Description

  • Issue: Advanced Configuration Pass-Through not working under pfSense > Services > Suricata > Edit Interface Settings - WAN (I'm using the WAN interface)
    Pfsense Version: 2.3.2-Release
    Suricata Version: 3.1.2_2
Reproduction:
  • Add the Suricata Service
  • Edit either of the two .yaml files available in the shell (as root)
    find / -name '*.yaml'

    /usr/local/etc/suricata/suricata.yaml
    /usr/local/etc/suricata/suricata_20934_ix1/suricata.yaml
    
  • Edit with vi, save.
  • Reload Suricata
    Suricata reloads, and rebuilds configuration files from Pfsense options (notice the time stamps):
    rwxr-xr-x 3 root wheel 512 Feb 18 02:04 .
    drwxr-xr-x 4 root wheel 512 Feb 18 02:04 ..
    rw-r--r- 1 root wheel 2888 Feb 18 16:49 classification.config
    rw-r--r- 1 root wheel 185 Feb 18 16:49 passlist
    rw-r--r- 1 root wheel 1332 Feb 18 16:49 reference.config
    drwxr-xr-x 2 root wheel 512 Feb 18 02:04 rules
    rw-r--r- 1 root wheel 2485735 Feb 18 16:49 sid-msg.map
    ** rw-r--r- 1 root wheel 8927 Feb 18 16:49 suricata.yaml**
    rw-r--r- 1 root wheel 0 Feb 18 16:49 threshold.config
    rw-r--r- 1 root wheel 53841 Feb 18 16:49 unicode.map
    
    drwxr-xr-x 3 root wheel 512 Feb 18 02:04 .
    drwxr-xr-x 4 root wheel 512 Feb 18 02:04 ..
    rw-r--r- 1 root wheel 2888 Feb 18 17:10 classification.config
    rw-r--r- 1 root wheel 185 Feb 18 17:10 passlist
    rw-r--r- 1 root wheel 1332 Feb 18 17:10 reference.config
    drwxr-xr-x 2 root wheel 512 Feb 18 02:04 rules
    rw-r--r- 1 root wheel 2485735 Feb 18 17:10 sid-msg.map
    **rw-r--r- 1 root wheel 8927 *Feb 18 17:10 suricata.yaml**
    rw-r--r- 1 root wheel 0 Feb 18 17:10 threshold.config
    rw-r--r- 1 root wheel 53841 Feb 18 17:10 unicode.map
    
* Check the loaded configuration: ps auxwww | grep suricata

root 52501 0.1 1.3 561304 418060 - Ss 5:10PM 0:11.72 /usr/local/bin/suricata -i ix1 -D -c /usr/local/etc/suricata/suricata_20934_ix1/suricata.yaml --pidfile /var/run/suricata_ix120934.pid
  • Contents of Advanced Configuration Pass-Through not parsed into the new suricata.yaml configuration file, after reload
  • Add the configuration to Services > Suricata > Edit Interface Settings - WAN (I'm using the WAN interface)> Advanced Configuration Pass- Through
  • Recheck the /usr/local/etc/suricata/suricata_20934_ix1/suricata.yaml file
    cat /usr/local/etc/suricata/suricata_20934_ix1/suricata.yaml
    
  • The added configuration does not load the Advanced Configuration Pass-Through contents.

This what I have in Advanced Configuration Pass - Through:

threading:
set-cpu-affinity: yes
- management-cpu-set:
cpu: [ 0 ] # include only these cpus in affinity settings
- receive-cpu-set:
cpu: [ 1 ] # include only these cpus in affinity settings
- decode-cpu-set:
cpu: [ "2" ]
mode: "balanced" 
- stream-cpu-set:
cpu: [ "0-3" ]
- detect-cpu-set:
cpu: [ "4,6" ]
mode: "exclusive" # run detect threads in these cpus # Use explicitely 3 threads and don't compute number by using # detect-thread-ratio variable:
threads: 3
prio:
low: [ "0-3" ]
medium: [ "5-7" ]
default: "medium" 
- verdict-cpu-set:
cpu: [ 0 ]
prio:
default: "high" 
- reject-cpu-set:
cpu: [ 0 ]
prio:
default: "low" 
- output-cpu-set:
cpu: [ "0" ]
prio:
default: "medium" 

detect:
profile: custom
custom-values:
toclient-groups: 200
toserver-groups: 200
sgh-mpm-context: auto
inspection-recursion-limit: 3000

  • Notice the suricata.yml file actual contents attached (does not include the added configuration in Advanced Configuration Pass-Through

The first tune for cpu-affinity (threading) found here: https://home.regit.org/2011/01/optimizing-suricata-on-a-multicore-cpu/
The second tune for Memory found here: http://suricata.readthedocs.io/en/latest/performance/high-performance-config.html

Hardware:

hw.model: Intel(R) Xeon(R) CPU D-1518 @ 2.20GHz
hw.machine: amd64
hw.ncpu: 8
real memory = 34359738368 (32768 MB)
avail memory = 33147830272 (31612 MB)

Result:
Pfsense is not parsing commands in the pfSense > Services > Suricata > Edit Interface Settings Advanced Configuration Pass-Through input.

Affected:
  • Users unable to tune advanced features in the Suricata configuration for Branch/Office Hardware
Actions #4

Updated by Michael Strasner about 7 years ago

LMFAO~!

Is there a workaround you can suggest?

Thanks for the update!

Actions #5

Updated by Kill Bill about 7 years ago

Well the above should give you a hint on what to add where. LOL. :-P

This package is actively maintained by https://github.com/bmeeks8?tab=activity, so I'd rather wait a bit before messing with that myself. (Also, the YAML thing is way more sensitive than snort.conf when it comes to producing invalid config, not really keen on touching this beast...)

Actions #6

Updated by Bill Meeks about 7 years ago

I will make some time to check into this. I had not realized the Advanced Pass-Through code was missing in Suricata. It may have gotten lost during the Bootstrap conversion.

Bill

Actions #7

Updated by Julian Wecke almost 7 years ago

Hi all,

i just run into this bug as i was testing configs for an other feature i'm currently developing for suricata package. So i decided to quickly fix it. PR: https://github.com/pfsense/FreeBSD-ports/pull/364

Greetings,
Julian aka securitym0nkey

Actions #8

Updated by Renato Botelho over 6 years ago

  • Status changed from New to Feedback
  • Target version set to 2.3.4-p2

Merged, thanks!

Actions #9

Updated by Jim Pingle over 6 years ago

  • Status changed from Feedback to Resolved
  • Target version deleted (2.3.4-p2)
Actions

Also available in: Atom PDF