Project

General

Profile

Actions

Bug #15516

closed

Per rule byte counter values lost across a filter reload

Added by Steve Wheeler about 1 month ago. Updated 6 days ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Rules / NAT
Target version:
Start date:
Due date:
% Done:

0%

Estimated time:
Plus Target Version:
24.08
Release Notes:
Default
Affected Version:
2.7.2
Affected Architecture:

Description

In some situations the byte counter reported by pfctl is not retained correctly across a filter reload.

It appears to be dependent on the value stored. So 4414450379 Bytes is retained correctly but 4793436239 Bytes is not:

[24.03-RELEASE][admin@4200.stevew.lan]/root: pfctl -vvsr | grep -A3 @90
@90 pass in quick on igc3 reply-to (igc3 172.21.16.1) inet proto tcp from <WAN__NETWORK:1> to 172.21.16.11 port = 5201 flags S/SA keep state (if-bound) label "USER_RULE: Allow iperf" label "id:1716563048" ridentifier 1716563048
  [ Evaluations: 95        Packets: 6177197   Bytes: 4793436239  States: 2     ]
  [ Inserted: uid 0 pid 0 State Creations: 2     ]
  [ Last Active Time: Fri May 24 21:27:52 2024 ]
[24.03-RELEASE][admin@4200.stevew.lan]/root: /etc/rc.filter_configure
[24.03-RELEASE][admin@4200.stevew.lan]/root: pfctl -vvsr | grep -A3 @90
@90 pass in quick on igc3 reply-to (igc3 172.21.16.1) inet proto tcp from <WAN__NETWORK:1> to 172.21.16.11 port = 5201 flags S/SA keep state (if-bound) label "USER_RULE: Allow iperf" label "id:1716563048" ridentifier 1716563048
  [ Evaluations: 96        Packets: 6177197   Bytes: 498468943   States: 0     ]
  [ Inserted: uid 0 pid 0 State Creations: 0     ]
  [ Last Active Time: N/A ]

The packet count remains the same.

Tested in 24.03 amd64

It does not happen if you then pass more traffic raising the byte counter above the limit and then reload the filter a second time.

Actions #1

Updated by Steve Y about 1 month ago

Just to add another data point, as I mentioned in https://forum.netgate.com/topic/188336/firewall-rule-counters-max-size/17 a counter of 96 GB was reset but a few at 5+ GB were not.

Actions #2

Updated by Jim Pingle 29 days ago

  • Plus Target Version changed from 24.07 to 24.08
Actions #3

Updated by Kristof Provost 16 days ago

I believe I see what's happening.

When rules are re-loaded we explicitly copy the counter state from the previous rules to the new rules. This uses the pf_counter_u64_add_protected() function (specifically introduced to work around counter_64 performance issues on 32-bit platforms, i.e. 3100), but this function takes a uint32_t as an argument.
So whenever our counter is over 32-bits we end up losing the upper bits.

This is masked to some extent because the internal counters are separated between in- and out-bound counters, but those get added up in userspace before they're displayed to users.

Actions #4

Updated by Mateusz Guzik 16 days ago

  • Assignee set to Mateusz Guzik

I agree with the analysis.

The entire thing however showed up because of performance problems on the 3100.

It very much can be addressed with modest hassle while retaining everything, which I'm going to sort out later today or tomorrow.

Actions #5

Updated by Mateusz Guzik 6 days ago

  • Status changed from New to Feedback

Fixed in https://gitlab.netgate.com/pfSense/FreeBSD-src/-/commit/5c71199c72c42a9836ed6f36521a7ff95e169e0b

Steve and Kristof verified the 3100 and amd64 respectively.

Actions #6

Updated by Steve Wheeler 6 days ago

Looks good with the patched kernel:

[24.08-DEVELOPMENT][admin@3100.stevew.lan]/root: pfctl -vvsr | grep -A3 @88
@88 pass in quick on mvneta1.4084 reply-to (mvneta1.4084 172.21.16.1) inet all flags S/SA keep state (if-bound) label "USER_RULE: Allow all ipv4+ipv6 via pfSsh.php" label "id:1652894570" ridentifier 1652894570
  [ Evaluations: 360       Packets: 7086185   Bytes: 5491882806  States: 7     ]
  [ Inserted: uid 0 pid 0 State Creations: 11    ]
  [ Last Active Time: Thu Jun 20 17:48:03 2024 ]
[24.08-DEVELOPMENT][admin@3100.stevew.lan]/root: /etc/rc.filter_configure
[24.08-DEVELOPMENT][admin@3100.stevew.lan]/root: pfctl -vvsr | grep -A3 @88
@88 pass in quick on mvneta1.4084 reply-to (mvneta1.4084 172.21.16.1) inet all flags S/SA keep state (if-bound) label "USER_RULE: Allow all ipv4+ipv6 via pfSsh.php" label "id:1652894570" ridentifier 1652894570
  [ Evaluations: 366       Packets: 7086207   Bytes: 5491885050  States: 0     ]
  [ Inserted: uid 0 pid 0 State Creations: 0     ]
  [ Last Active Time: N/A ]

Actions #7

Updated by Marcos M 6 days ago

  • Status changed from Feedback to Resolved
Actions

Also available in: Atom PDF