Project

General

Profile

Actions
Copy link

Feature #15038

closed

Operating System support for PF ``pflow`` packet data flow export

Added by Jim Pingle about 1 year ago. Updated 10 months ago.

Status:
Closed
Priority:
Normal
Assignee:
Kristof Provost
Category:
Operating System
Target version:
24.03
Start date:
Due date:
% Done:

100%

Estimated time:
Release Notes:
Default

Description

Import the pflow netflow/IPFIX export functionality for PF from OpenBSD into pfSense Plus

Related issues

Precedes Feature #15039: GUI to configure Packet Flow Data (``pflow``) exportClosedJim Pingle

Actions
Actions
Copy link
 #1

Updated by Jim Pingle about 1 year ago

  • Precedes Feature #15039: GUI to configure Packet Flow Data (``pflow``) export added
Actions
Copy link
 #2

Updated by Kristof Provost about 1 year ago

The plan is to start by importing OpenBSD's pflow(4), and then extending it to expose more NAT information (see RFC 8158), as well as millisecond-level timestamps.

The latter of these will likely require more pf-internal changes, because the creation timestamp is currently tracked at second level granularity.

Actions
Copy link
 #3

Updated by Kristof Provost 11 months ago

  • Status changed from New to Feedback

I've merged the upstream pflow code. It'll be part of the next snapshot builds.

It includes support for netflow v5, IPFX (i.e. v10), sub-second accurate timestamps and RFC 8158 NAT44 information. I'll add a brief description on how to use it in #15039

Actions
Copy link
 #4

Updated by Jim Pingle 11 months ago

  • Status changed from Feedback to In Progress

On 24.03.a.20240117.0600 which should be after this was merged, pflowctl complaints that pflow.ko is not loaded, but that file is not present in the build nor is it in the kernel configuration.

: pflowctl -l
pflowctl: pflow.ko is not loaded.
: find / -name pflow.ko
: 
: sysctl kern.conftxt | grep -i flow
:
Actions
Copy link
 #5

Updated by Kristof Provost 11 months ago

I forgot to include the module in the image in cross build. That's done now, so the next build will have it.

Actions
Copy link
 #6

Updated by Jim Pingle 11 months ago

  • Status changed from In Progress to Feedback
  • % Done changed from 0 to 100

Module is present now on latest build (24.03.a.20240122.0600) and loads OK.

Had a weird glitch once where I couldn't pass traffic to a test host after configuring pflow0 but thus far I haven't been able to reproduce it. If I can figure out what happened there I'll post more details.

With the state default set to pflow and a manual config, I am seeing flows making it through to a collector.

Actions
Copy link
 #7

Updated by Jim Pingle 11 months ago

  • Status changed from Feedback to In Progress
  • % Done changed from 100 to 90

After testing a bit I discovered what might be a minor issue.

Not that I expect anyone to need to go this high, but the observation domain on the pflowctl command is using a signed 32-bit int and I believe it should be unsigned 32-bit int. If I try to set a value > 2147483647 it shows negative values in pflowctl -l.

On TNSR in the Yang spec we appear to set it as a u32 as well. Not sure what it was on OpenBSD.

For now I can have whatever input I allow to be capped at 2147483647

Actions
Copy link
 #8

Updated by Jim Pingle 11 months ago

Per Kristof, the above was an output printing issue in pflowctl and it was correct internally. That will be fixed soon and have a test case added.

Next issue I've found is that despite setting set state-defaults pflow, an option such as sloppy on a rule clobbers the default pflow setting:

rules.debug:
pass  in  quick  on $LAN4 inet proto tcp  from any to any ridentifier 1705931230 flags S/SA keep state ( sloppy  ) label "USER_RULE: Sloppy Test" label "id:1705931230" 

pfctl -vvsr:
@175 pass in quick on igc3 inet proto tcp all flags S/SA keep state (if-bound, sloppy) label "USER_RULE: Sloppy Test" label "id:1705931230" ridentifier 1705931230

However if I manually add in pflow then both appear:

rules.debug:
pass  in  quick  on $LAN4 inet proto tcp  from any to any ridentifier 1705931230 flags S/SA keep state ( pflow, sloppy  ) label "USER_RULE: Sloppy Test" label "id:1705931230" 

pfctl -vvsr:
@175 pass in quick on igc3 inet proto tcp all flags S/SA keep state (if-bound, sloppy, pflow) label "USER_RULE: Sloppy Test" label "id:1705931230" ridentifier 1705931230
Actions
Copy link
 #9

Updated by Kristof Provost 11 months ago

  • Status changed from In Progress to Feedback

The signed vs. unsigned issue will be fixed in the next snapshot.

The issue in comment 8 is actually expected behaviour. set state-defaults pflow sets the default that applies is no other state flags are set on the rule. As soon as a flag (e.g. sloppy) is set the rule does not follow the default flags, but only the ones set in the rule itself.

Actions
Copy link
 #10

Updated by Jim Pingle 10 months ago

  • Status changed from Feedback to Closed
  • % Done changed from 90 to 100

Looks good on current snapshots, everything appears to be working as expected.

: pflowctl -c
pflow1
: pflowctl -s pflow1 dst 172.21.32.3:2056 domain 2149999999
: pflowctl -l
pflow1: version 5 domain 2149999999 dst 172.21.32.3:2056
Actions
Copy link
 #11

Updated by Jim Pingle 10 months ago

  • Subject changed from Support for ``pflow`` IPFIX export to Operating System support for PF ``pflow`` packet data flow export
Actions
Copy link

Also available in: Atom PDF