Project

General

Profile

Actions

Feature #13382

closed

Packet Capture GUI with granular control

Added by Marcos M almost 2 years ago. Updated about 1 year ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Packet Capture
Target version:
Start date:
Due date:
% Done:

100%

Estimated time:
Plus Target Version:
23.05
Release Notes:
Default

Description

This is a complete re-write of the Packet Capture page.

Changes:
- Saved filename now includes the interface and start time.
- Promiscuous mode is enabled by default.
- Default packet count is set to 1000 instead of 100.

Enhancements:
- Automatically convert IP subnet input to a valid subnet to avoid the "non-network bits set" error. This allows the input of e.g. 10.0.0.1/24.
- Allow stopping a running capture even with input errors in the form.
- Avoid writing an extra file by reading the last capture start time from the file name.
- Keep previous packet captures on the disk (removed when the Clear button is clicked).

New:
- Quick filter presets for tagged/untagged traffic.
- Allow matching on multiple type values, e.g. tcp or udp.
- Allow setting custom values for all types, e.g. ether proto 0x88cc.
- Allow filtering for VLAN tags, and the tag level (for stacked VLAN packets).
- Allow filtering on interfaces that are not assigned (e.g. for VLAN parent interfaces).
- Allow forcing any view type, e.g. for CARP traffic.
- Live preview of the packet capture.

Missing (for future MR):
- Run multiple packet captures at the same time.
- Input validation for all filters that would reject all packets, such as 'icmp6 and ip'.


Files

2023-04-05_10-59.png (138 KB) 2023-04-05_10-59.png Jim Pingle, 04/05/2023 10:04 AM

Related issues

Related to Feature #13322: Define Packet Capture ProtocolClosedMarcos M

Actions
Related to Feature #13094: Allow packet capture filtering in tagged packetsClosedMarcos M

Actions
Related to Feature #13411: Packet capture does not support 6rd tunnelsClosedMarcos M

Actions
Related to Feature #13682: Automatically indicate a packet capture has stopped when count limit is reachedClosedMarcos M

Actions
Related to Feature #13868: Allow packet capture on unassigned interfacesClosedMarcos M

Actions
Related to Feature #13017: Packet capture: add preview results while capture is runningClosedMarcos M

Actions
Actions #1

Updated by Marcos M almost 2 years ago

Actions #2

Updated by Marcos M almost 2 years ago

  • Related to Feature #13094: Allow packet capture filtering in tagged packets added
Actions #3

Updated by Marcos M almost 2 years ago

  • Status changed from New to In Progress
Actions #4

Updated by Marcos M almost 2 years ago

  • Target version set to 2.7.0
  • Plus Target Version set to 22.11
Actions #5

Updated by Christopher Cope almost 2 years ago

Promiscuous mode is on by default, as compared to previously where it is off by default, and turning it off doesn't seem to work.

When I turn it off and then click Start it turns back on and the command ran doesn't change.

Actions #6

Updated by Marcos M almost 2 years ago

It's now fixed. Since it's currently still a work in progress, please leave feedback on the MR page if you have access - thanks!

Actions #7

Updated by Louis B almost 2 years ago

Sometimes, I would like to monitor what is happening on multiple vlans = interfaces at the same time. So I would be glad to see an option to start a packet capture on interface-1, and keep that one running when switching to inteface-2 and start another one.

Perhaps I can do that by starting two pfSense gui instances in parallel, however I am not sure that works.

Actions #8

Updated by Marcos M almost 2 years ago

  • Description updated (diff)
  • Status changed from In Progress to Pull Request Review
  • % Done changed from 0 to 100

Louis B wrote in #note-7:

Sometimes, I would like to monitor what is happening on multiple vlans = interfaces at the same time. So I would be glad to see an option to start a packet capture on interface-1, and keep that one running when switching to inteface-2 and start another one.

Perhaps I can do that by starting two pfSense gui instances in parallel, however I am not sure that works.

For now, this can be achieved by capturing on the VLAN parent interface (given the VLANs are on the same interface).

Actions #9

Updated by Marcos M almost 2 years ago

  • File 838.diff added
Actions #10

Updated by Marcos M almost 2 years ago

  • File deleted (838.diff)
Actions #11

Updated by Marcos M almost 2 years ago

  • File 838.diff added
  • Description updated (diff)
Actions #12

Updated by Marcos M almost 2 years ago

  • Description updated (diff)
Actions #13

Updated by Marcos M almost 2 years ago

  • File deleted (838.diff)
Actions #14

Updated by Marcos M almost 2 years ago

  • File 838.diff added
Actions #15

Updated by Marcos M almost 2 years ago

  • Related to Feature #13411: Packet capture does not support 6rd tunnels added
Actions #16

Updated by Jim Pingle almost 2 years ago

  • Plus Target Version changed from 22.11 to 23.01
Actions #17

Updated by Jim Pingle over 1 year ago

  • Plus Target Version changed from 23.01 to 23.05
Actions #18

Updated by Marcos M over 1 year ago

  • Related to Feature #13682: Automatically indicate a packet capture has stopped when count limit is reached added
Actions #19

Updated by Marcos M over 1 year ago

  • Description updated (diff)
Actions #20

Updated by Marcos M over 1 year ago

  • File deleted (838.diff)
Actions #21

Updated by Marcos M over 1 year ago

  • Related to Feature #13868: Allow packet capture on unassigned interfaces added
Actions #22

Updated by Marcos M over 1 year ago

  • Related to Feature #13017: Packet capture: add preview results while capture is running added
Actions #23

Updated by Marcos M over 1 year ago

  • Status changed from Pull Request Review to Feedback
Actions #24

Updated by Christian McDonald over 1 year ago

We are going to need updated docs and a better subject line for release notes.

Actions #25

Updated by Marcos M over 1 year ago

  • Subject changed from More flexible packet capture GUI to Packet Capture GUI with granular control
  • Description updated (diff)
Actions #26

Updated by Jim Pingle over 1 year ago

Spent a couple minutes with the new page and hit a few snags:

  • It's kind of confusing having none of the filtering options visible by default, probably going to get a lot of questions about that. At least consider moving "Custom Filter" up under "Everything" and consider making the filtering drop-down a radio selector instead so the available options are much more obvious.
  • It doesn't seem possible to do an "and". There are two "any of" types ("any of", "any of [OR]") but they both seem to do an 'or' in the filter when running the command (the command being run is identical and uses 'or') so I'm not sure what the difference is between those two options. For example in the old UI I could do 198.51.100.7,224.0.0.18 and it would capture only traffic where those two hosts were involved either as src->dst or dst->src. A brief look at the code seems to indicate "any of" should be "all of [AND]" since a logical and is not "any of", but something isn't working there since it's not actually doing an 'and' operation even when that is selected. The test I ran was with trying to filter two untagged host addresses. See the attached image. This is on the latest snapshot.
  • When viewing a capture of CARP, it is not using the CARP format output (-T carp) as it did on the old page so it's misinterpreting it as VRRPv2 in the output. This is very important when diagnosing CARP issues.

Old:

10:34:43.227163 00:00:5e:00:01:4d > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 70: (tos 0xe0, ttl 255, id 0, offset 0, flags [DF], proto VRRP (112), length 56)
    198.51.100.7 > 224.0.0.18: carp 198.51.100.7 > 224.0.0.18: CARPv2-advertise 36: vhid=77 advbase=1 advskew=0 authlen=7 counter=6443679781330698894

New:

14:33:52.724603 00:00:5e:00:01:4d > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 70: (tos 0xe0, ttl 255, id 0, offset 0, flags [DF], proto VRRP (112), length 56)
    198.51.100.7 > 224.0.0.18: vrrp 198.51.100.7 > 224.0.0.18: VRRPv2, Advertisement, vrid 77, prio 0, authtype none, intvl 1s, length 36, addrs(7): 226.182.73.11,204.65.161.56,200.77.0.155,251.188.228.33,75.158.60.199,151.217.255.250,251.182.184.229
Actions #27

Updated by Marcos M over 1 year ago

  • Status changed from New to In Progress
Actions #28

Updated by Marcos M over 1 year ago

  • Status changed from In Progress to Pull Request Review

https://gitlab.netgate.com/pfSense/pfSense/-/merge_requests/1025

  • Moved custom filter option to the first position in the list
  • Add the option "all of" and "[OR] all of" for supported Types (host, mac, port)
  • Add the "View Type" option to force a specific traffic type, e.g. for CARP
Actions #29

Updated by Marcos M over 1 year ago

  • Status changed from Pull Request Review to Feedback
  • % Done changed from 0 to 100
Actions #30

Updated by Alhusein Zawi over 1 year ago

Is there way to keep/add the classic view of the packet capture ?
in many cases I need to start a quick pcap by selecting 1 protocol (like icmp) without adding any filter.

Actions #31

Updated by Jim Pingle over 1 year ago

  • Status changed from Feedback to Pull Request Review
Actions #32

Updated by Marcos M over 1 year ago

  • Status changed from Pull Request Review to Feedback
Actions #33

Updated by Marcos M over 1 year ago

  • Description updated (diff)

Alhusein Zawi wrote in #note-30:

Is there way to keep/add the classic view of the packet capture ?
in many cases I need to start a quick pcap by selecting 1 protocol (like icmp) without adding any filter.

The latest changes simplify the page and keeps the Custom Filter Options panel shown by default.

Actions #34

Updated by Jim Pingle over 1 year ago

  • Status changed from Feedback to Pull Request Review
Actions #35

Updated by Marcos M over 1 year ago

  • Status changed from Pull Request Review to Feedback
Actions #36

Updated by Jordan G over 1 year ago

23.05.a.20230421.0022 packet capture has the updated interface with additional options but now also has the filter preset which can be changed from custom filter to remove the additional options and provide a simplified basic option set for packet capture.

feedback - I don't know if its absolutely necessary to have punctuation for the first two field's description, "Interface to capture packets on." and "Filter preset." ; it's odd looking albeit probably grammatically proper

Actions #37

Updated by Jim Pingle about 1 year ago

  • Status changed from Feedback to New
  • % Done changed from 100 to 90

There seems to be an issue with the logic on the page now. If you only want to filter based on the top section of custom filter options and leave the bottom section as-is, it always places or ((not vlan)) in the command line which makes the first part of the filter ineffective since it will match anything without a VLAN tag.

For example if you pick CARP from the protocol drop-down and click start, it uses:

/usr/sbin/tcpdump -ni vtnet1 -c 1000 -U -w - '((proto 112)) or ((not vlan))'

Thus it's matching any packet on that interface without a VLAN tag, which in this case is everything.

Similarly if you switch that tagged filter section option to "include any of" it adds or ((vlan)).

At least in the first case it seems like it should be and ((not vlan)), not or. The second case is less clear.

Actions #38

Updated by Marcos M about 1 year ago

  • Status changed from New to Pull Request Review

Thanks! https://gitlab.netgate.com/pfSense/pfSense/-/merge_requests/1032/diffs addresses that issue. This worked with every combination I could think of, including different tag levels, exclusions, and type presets.

Similarly if you switch that tagged filter section option to "include any of" it adds or ((vlan)).

This one is intended; it's needed to include both untagged and tagged packets at the same time (filtered as needed given the input). Using and vlan would force the match on a packet to be simultaneously tagged/untagged and would fail.

Actions #39

Updated by Jim Pingle about 1 year ago

  • Status changed from Pull Request Review to Feedback
  • % Done changed from 90 to 100

MRs merged, ready to be tested again.

Actions #40

Updated by Jim Pingle about 1 year ago

Latest code looks better to me. I can filter using the top set of options and it doesn't grab more than I asked for.

Will leave open for additional feedback for now, though.

Actions #41

Updated by Jim Pingle about 1 year ago

  • Status changed from Feedback to Resolved

Still seems to be OK at least with the light testing I've done. If any new problems come up they can go into separate Redmine issues.

Actions #42

Updated by Jim Pingle about 1 year ago

  • Category changed from Diagnostics to Packet Capture
Actions

Also available in: Atom PDF