Regression #13966


RRD update script does not parse state data properly

Added by Jim Pingle 3 months ago. Updated about 2 months ago.

RRD Graphs
Target version:
Start date:
Due date:
% Done:


Estimated time:
Plus Target Version:
Release Notes:
Affected Version:
Affected Architecture:


The states graph is not functional on 23.01 and current dev snapshots of CE and Plus

The regex pattern used by the commands with egrep no longer works as it did in the past. The same command and data on an older host succeeds.

        $rrdupdatesh .= "pfctl_si_out=\"` $pfctl -si > /tmp/pfctl_si_out `\"\n";
        $rrdupdatesh .= "pfctl_ss_out=\"` $pfctl -ss > /tmp/pfctl_ss_out`\"\n";
        $rrdupdatesh .= "pfrate=\"` cat /tmp/pfctl_si_out | egrep \"inserts|removals\" | awk '{ pfrate = \$3 + pfrate } {print pfrate}'|tail -1 `\"\n";
        $rrdupdatesh .= "pfstates=\"` cat /tmp/pfctl_ss_out | egrep -v \"<\\-.*?<\\-|\\->.*?\\->\" | wc -l|sed 's/ //g'`\"\n";
        $rrdupdatesh .= "pfnat=\"` cat /tmp/pfctl_ss_out | egrep '<\\-.*?<\\-|\\->.*?\\->' | wc -l|sed 's/ //g' `\"\n";
        $rrdupdatesh .= "srcip=\"` cat /tmp/pfctl_ss_out | egrep -v '<\\-.*?<\\-|\\->.*?\\->' | grep '\\->' | awk '{print \$3}' | awk -F: '{print \$1}' | sort -u|wc -l|sed 's/ //g' `\"\n";
        $rrdupdatesh .= "dstip=\"` cat /tmp/pfctl_ss_out | egrep -v '<\\-.*?<\\-|\\->.*?\\->' | grep '<\\-' | awk '{print \$3}' | awk -F: '{print \$1}' | sort -u|wc -l|sed 's/ //g' `\"\n";

The "rate" line works, the four after it fail with the following error:

egrep: repetition-operator operand invalid

It also fails if you try to run the script line manually from a shell prompt:

: cat /tmp/pfctl_ss_out | egrep '<\-.*?<\-|\->.*?\->' | wc -l | sed 's/ //g'
egrep: repetition-operator operand invalid

The same command works on 22.05 even with the data file from 23.01, so it's definitely an issue with the regex.

Actions #1

Updated by Reid Linnemann 3 months ago

  • Assignee set to Reid Linnemann

This has probably not worked correctly in a while. In addition to '.*?' being ambiguous in extended REs, these patterns don't match current pfctl output for NAT entries.

The proper posix extended regex for matching nat rules would be

\(([0-9a-f:.]|\[|\])+\) (\->|<\-)

as NAT entries are indicated by an address enclosed in parens.

Actions #2

Updated by Reid Linnemann 3 months ago

  • Status changed from New to Feedback
  • % Done changed from 0 to 100
Actions #3

Updated by Azamat Khakimyanov about 2 months ago

  • Status changed from Feedback to Resolved

Tested on 23.01 and on 23.05-DEV (built on Fri Apr 07 01:20:44 UTC 2023) and on 2.7-DEV (built on Wed Apr 12 06:05:24 UTC 2023)

I was able to reproduce this issue on 23.01 - it didn't matter how many web pages I opened on my laptop behind pfSense, all (Filter states, Source addr. and dest. addr.) counters were zero (empty)

On 23.05-DEV and on 2.7-DEV I didn't see this issue. When I opened a lot of web pages I saw that filter state and NAT states showed 800+ states as Maximum, Average and Last counters were not zero too.

I marked this Regression as Resolved.


Also available in: Atom PDF