Project

General

Profile

Bug #7278

Suricata Service - Advanced Configuration Pass-Through not working

Added by Michael Strasner over 2 years ago. Updated almost 2 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
-
Category:
Suricata
Target version:
-
Start date:
02/18/2017
Due date:
% Done:

0%

Estimated time:
Affected Version:
2.3.2
Affected Architecture:

Description

  • Issue: Advanced Configuration Pass-Through not working under pfSense > Services > Suricata > Edit Interface Settings - WAN (I'm using the WAN interface)

Pfsense Version: 2.3.2-Release
Suricata Version: 3.1.2_2

  • Reproduction:
  • Add the Suricata Service
  • Edit either of the two .yaml files available in the shell (as root)
  • find / -name '*.yaml'

/usr/local/etc/suricata/suricata.yaml
/usr/local/etc/suricata/suricata_20934_ix1/suricata.yaml

  • Edit with vi, save.
  • Reload Suricata
    Suricata reloads, and rebuilds configuration files from Pfsense options (notice the time stamps):

rwxr-xr-x 3 root wheel 512 Feb 18 02:04 .
drwxr-xr-x 4 root wheel 512 Feb 18 02:04 ..
rw-r--r- 1 root wheel 2888 Feb 18 16:49 classification.config
rw-r--r- 1 root wheel 185 Feb 18 16:49 passlist
rw-r--r- 1 root wheel 1332 Feb 18 16:49 reference.config
drwxr-xr-x 2 root wheel 512 Feb 18 02:04 rules
rw-r--r- 1 root wheel 2485735 Feb 18 16:49 sid-msg.map
rw-r--r- 1 root wheel 8927 Feb 18 16:49 suricata.yaml
rw-r--r- 1 root wheel 0 Feb 18 16:49 threshold.config
rw-r--r- 1 root wheel 53841 Feb 18 16:49 unicode.map

drwxr-xr-x 3 root wheel 512 Feb 18 02:04 .
drwxr-xr-x 4 root wheel 512 Feb 18 02:04 ..
rw-r--r- 1 root wheel 2888 Feb 18 17:10 classification.config
rw-r--r- 1 root wheel 185 Feb 18 17:10 passlist
rw-r--r- 1 root wheel 1332 Feb 18 17:10 reference.config
drwxr-xr-x 2 root wheel 512 Feb 18 02:04 rules
rw-r--r- 1 root wheel 2485735 Feb 18 17:10 sid-msg.map
rw-r--r- 1 root wheel 8927 Feb 18 17:10 suricata.yaml
rw-r--r- 1 root wheel 0 Feb 18 17:10 threshold.config
rw-r--r- 1 root wheel 53841 Feb 18 17:10 unicode.map

  • Check the loaded configuration: ps auxwww | grep suricata

root 52501 0.1 1.3 561304 418060 - Ss 5:10PM 0:11.72 /usr/local/bin/suricata -i ix1 -D -c /usr/local/etc/suricata/suricata_20934_ix1/suricata.yaml --pidfile /var/run/suricata_ix120934.pid

Contents of Advanced Configuration Pass-Through not parsed into the new suricata.yaml configuration file, after reload

  • Add the configuration to Services > Suricata > Edit Interface Settings - WAN (I'm using the WAN interface)> Advanced Configuration Pass- Through
  • Recheck the /usr/local/etc/suricata/suricata_20934_ix1/suricata.yaml file
    The added configuration does not load the Advanced Configuration Pass-Through contents (this what I have in Advanced Configuration Pass - Through):

threading:
set-cpu-affinity: yes
- management-cpu-set:
cpu: [ 0 ] # include only these cpus in affinity settings
- receive-cpu-set:
cpu: [ 1 ] # include only these cpus in affinity settings
- decode-cpu-set:
cpu: [ "2" ]
mode: "balanced"
- stream-cpu-set:
cpu: [ "0-3" ]
- detect-cpu-set:
cpu: [ "4,6" ]
mode: "exclusive" # run detect threads in these cpus # Use explicitely 3 threads and don't compute number by using # detect-thread-ratio variable:
threads: 3
prio:
low: [ "0-3" ]
medium: [ "5-7" ]
default: "medium"
- verdict-cpu-set:
cpu: [ 0 ]
prio:
default: "high"
- reject-cpu-set:
cpu: [ 0 ]
prio:
default: "low"
- output-cpu-set:
cpu: [ "0" ]
prio:
default: "medium"

detect:
profile: custom
custom-values:
toclient-groups: 200
toserver-groups: 200
sgh-mpm-context: auto
inspection-recursion-limit: 3000

Notice the suricata.yml file actual contents attached (does not include the added configuration in Advanced Configuration Pass-Through

The first tune for cpu-affinity (threading) found here: https://home.regit.org/2011/01/optimizing-suricata-on-a-multicore-cpu/
The second tune for Memory found here: http://suricata.readthedocs.io/en/latest/performance/high-performance-config.html

  • Hardware:
    I have low power server Xeon, with high memory seeking to tune Suricata (set and then forget, basically)

hw.model: Intel(R) Xeon(R) CPU D-1518 @ 2.20GHz
hw.machine: amd64
hw.ncpu: 8
real memory = 34359738368 (32768 MB)
avail memory = 33147830272 (31612 MB)

  • Result:
    Pfsense is not parsing the
    Advanced Configuration Pass-Through.
  • Affected: Unable to tune advanced features in the Suricata configuration for Branch/Office Hardware
suricata.yaml (8.72 KB) suricata.yaml Suricata YML does not receive Advanced Configuration Pass-Through data Michael Strasner, 02/18/2017 04:32 PM

History

#1 Updated by Kill Bill over 2 years ago

Please, use the pre button to post code/command output. This is just unreadable mess.

#2 Updated by Michael Strasner over 2 years ago

This is just ... mess.

Interesting wording, that's what I thought of the feature.

Description

  • Issue: Advanced Configuration Pass-Through not working under pfSense > Services > Suricata > Edit Interface Settings - WAN (I'm using the WAN interface)
    Pfsense Version: 2.3.2-Release
    Suricata Version: 3.1.2_2
Reproduction:
  • Add the Suricata Service
  • Edit either of the two .yaml files available in the shell (as root)
    find / -name '*.yaml'

    /usr/local/etc/suricata/suricata.yaml
    /usr/local/etc/suricata/suricata_20934_ix1/suricata.yaml
    
  • Edit with vi, save.
  • Reload Suricata
    Suricata reloads, and rebuilds configuration files from Pfsense options (notice the time stamps):
    rwxr-xr-x 3 root wheel 512 Feb 18 02:04 .
    drwxr-xr-x 4 root wheel 512 Feb 18 02:04 ..
    rw-r--r- 1 root wheel 2888 Feb 18 16:49 classification.config
    rw-r--r- 1 root wheel 185 Feb 18 16:49 passlist
    rw-r--r- 1 root wheel 1332 Feb 18 16:49 reference.config
    drwxr-xr-x 2 root wheel 512 Feb 18 02:04 rules
    rw-r--r- 1 root wheel 2485735 Feb 18 16:49 sid-msg.map
    ** rw-r--r- 1 root wheel 8927 Feb 18 16:49 suricata.yaml**
    rw-r--r- 1 root wheel 0 Feb 18 16:49 threshold.config
    rw-r--r- 1 root wheel 53841 Feb 18 16:49 unicode.map
    
    drwxr-xr-x 3 root wheel 512 Feb 18 02:04 .
    drwxr-xr-x 4 root wheel 512 Feb 18 02:04 ..
    rw-r--r- 1 root wheel 2888 Feb 18 17:10 classification.config
    rw-r--r- 1 root wheel 185 Feb 18 17:10 passlist
    rw-r--r- 1 root wheel 1332 Feb 18 17:10 reference.config
    drwxr-xr-x 2 root wheel 512 Feb 18 02:04 rules
    rw-r--r- 1 root wheel 2485735 Feb 18 17:10 sid-msg.map
    **rw-r--r- 1 root wheel 8927 *Feb 18 17:10 suricata.yaml**
    rw-r--r- 1 root wheel 0 Feb 18 17:10 threshold.config
    rw-r--r- 1 root wheel 53841 Feb 18 17:10 unicode.map
    
* Check the loaded configuration: ps auxwww | grep suricata

root 52501 0.1 1.3 561304 418060 - Ss 5:10PM 0:11.72 /usr/local/bin/suricata -i ix1 -D -c /usr/local/etc/suricata/suricata_20934_ix1/suricata.yaml --pidfile /var/run/suricata_ix120934.pid
  • Contents of Advanced Configuration Pass-Through not parsed into the new suricata.yaml configuration file, after reload
  • Add the configuration to Services > Suricata > Edit Interface Settings - WAN (I'm using the WAN interface)> Advanced Configuration Pass- Through
  • Recheck the /usr/local/etc/suricata/suricata_20934_ix1/suricata.yaml file
    cat /usr/local/etc/suricata/suricata_20934_ix1/suricata.yaml
    
  • The added configuration does not load the Advanced Configuration Pass-Through contents.

This what I have in Advanced Configuration Pass - Through:

threading:
set-cpu-affinity: yes
- management-cpu-set:
cpu: [ 0 ] # include only these cpus in affinity settings
- receive-cpu-set:
cpu: [ 1 ] # include only these cpus in affinity settings
- decode-cpu-set:
cpu: [ "2" ]
mode: "balanced" 
- stream-cpu-set:
cpu: [ "0-3" ]
- detect-cpu-set:
cpu: [ "4,6" ]
mode: "exclusive" # run detect threads in these cpus # Use explicitely 3 threads and don't compute number by using # detect-thread-ratio variable:
threads: 3
prio:
low: [ "0-3" ]
medium: [ "5-7" ]
default: "medium" 
- verdict-cpu-set:
cpu: [ 0 ]
prio:
default: "high" 
- reject-cpu-set:
cpu: [ 0 ]
prio:
default: "low" 
- output-cpu-set:
cpu: [ "0" ]
prio:
default: "medium" 

detect:
profile: custom
custom-values:
toclient-groups: 200
toserver-groups: 200
sgh-mpm-context: auto
inspection-recursion-limit: 3000

  • Notice the suricata.yml file actual contents attached (does not include the added configuration in Advanced Configuration Pass-Through

The first tune for cpu-affinity (threading) found here: https://home.regit.org/2011/01/optimizing-suricata-on-a-multicore-cpu/
The second tune for Memory found here: http://suricata.readthedocs.io/en/latest/performance/high-performance-config.html

Hardware:

hw.model: Intel(R) Xeon(R) CPU D-1518 @ 2.20GHz
hw.machine: amd64
hw.ncpu: 8
real memory = 34359738368 (32768 MB)
avail memory = 33147830272 (31612 MB)

Result:
Pfsense is not parsing commands in the pfSense > Services > Suricata > Edit Interface Settings Advanced Configuration Pass-Through input.

Affected:
  • Users unable to tune advanced features in the Suricata configuration for Branch/Office Hardware

#4 Updated by Michael Strasner over 2 years ago

LMFAO~!

Is there a workaround you can suggest?

Thanks for the update!

#5 Updated by Kill Bill over 2 years ago

Well the above should give you a hint on what to add where. LOL. :-P

This package is actively maintained by https://github.com/bmeeks8?tab=activity, so I'd rather wait a bit before messing with that myself. (Also, the YAML thing is way more sensitive than snort.conf when it comes to producing invalid config, not really keen on touching this beast...)

#6 Updated by Bill Meeks over 2 years ago

I will make some time to check into this. I had not realized the Advanced Pass-Through code was missing in Suricata. It may have gotten lost during the Bootstrap conversion.

Bill

#7 Updated by Julian Wecke about 2 years ago

Hi all,

i just run into this bug as i was testing configs for an other feature i'm currently developing for suricata package. So i decided to quickly fix it. PR: https://github.com/pfsense/FreeBSD-ports/pull/364

Greetings,
Julian aka securitym0nkey

#8 Updated by Renato Botelho about 2 years ago

  • Status changed from New to Feedback
  • Target version set to 2.3.4-p2

Merged, thanks!

#9 Updated by Jim Pingle almost 2 years ago

  • Status changed from Feedback to Resolved
  • Target version deleted (2.3.4-p2)

Also available in: Atom PDF