Project

General

Profile

Actions
Copy link

Feature #12963

open

Run nmap scans in the background

Added by Phil Wardt almost 3 years ago. Updated over 2 years ago.

Status:
Feedback
Priority:
Normal
Assignee:
Jim Pingle
Category:
Nmap
Target version:
-
Start date:
Due date:
% Done:

0%

Estimated time:
Plus Target Version:

Description

NMap package cannot actually run from gui because of nginx timeout

This patch adds the following features:
- run nmap in background and output results to /root/nmap.result file
- do not start a new nmap session from gui until last one is completed
- add a results tab to get the last log output from nmap
- show the time of last nmap scan completion (start time is displayed by actual command output)

A github commit was pushed and linked in next comment

Files

Download all files
02-scan-exec.JPG (70.6 KB) 02-scan-exec.JPG Scan executed Phil Wardt, 03/20/2022 08:22 AM
03-scan-wait.JPG (45.4 KB) 03-scan-wait.JPG Scan wait to finish Phil Wardt, 03/20/2022 08:22 AM
01-scan.JPG (272 KB) 01-scan.JPG Scan tab Phil Wardt, 03/20/2022 08:22 AM
04-results.JPG (231 KB) 04-results.JPG Results tab Phil Wardt, 03/20/2022 08:22 AM
nmap.patch (4.64 KB) nmap.patch Phil Wardt, 03/20/2022 08:48 AM
nmap-complete.patch (5.73 KB) nmap-complete.patch Phil Wardt, 03/21/2022 07:51 AM
04-results-refresh.JPG (137 KB) 04-results-refresh.JPG Phil Wardt, 03/21/2022 07:51 AM
05-results-complete.JPG (175 KB) 05-results-complete.JPG Phil Wardt, 03/21/2022 07:51 AM
06-results-none.JPG (120 KB) 06-results-none.JPG Phil Wardt, 03/21/2022 07:51 AM
nmap-complete.patch (5.74 KB) nmap-complete.patch Phil Wardt, 03/21/2022 04:55 PM
nmap-complete-v4-ok.patch (5.7 KB) nmap-complete-v4-ok.patch Phil Wardt, 03/22/2022 03:20 PM
07-results-complete-new.JPG (147 KB) 07-results-complete-new.JPG Phil Wardt, 03/22/2022 03:24 PM
nmap-complete-v5-ok.patch (6.26 KB) nmap-complete-v5-ok.patch Phil Wardt, 03/23/2022 09:29 AM
08-delete-icon.JPG (29.8 KB) 08-delete-icon.JPG Phil Wardt, 03/24/2022 02:21 PM
02-Wait-1.JPG (193 KB) 02-Wait-1.JPG Phil Wardt, 03/28/2022 05:02 AM
01-Start.JPG (234 KB) 01-Start.JPG Phil Wardt, 03/28/2022 05:02 AM
02-Wait-2.JPG (221 KB) 02-Wait-2.JPG Phil Wardt, 03/28/2022 05:02 AM
02-Wait-3.JPG (352 KB) 02-Wait-3.JPG Phil Wardt, 03/28/2022 05:02 AM
03-Results.JPG (345 KB) 03-Results.JPG Phil Wardt, 03/28/2022 05:02 AM
03-Results-errors-only.JPG (227 KB) 03-Results-errors-only.JPG Phil Wardt, 03/28/2022 05:02 AM
03-Results-with-errors.JPG (236 KB) 03-Results-with-errors.JPG Phil Wardt, 03/28/2022 05:02 AM
nmap_scan-v15.patch (14.7 KB) nmap_scan-v15.patch Phil Wardt, 03/28/2022 05:09 AM
01-Start-options-syntax.JPG (208 KB) 01-Start-options-syntax.JPG Phil Wardt, 03/31/2022 04:00 PM
02-Start-scan-methods.JPG (302 KB) 02-Start-scan-methods.JPG Phil Wardt, 03/31/2022 04:00 PM
03-Results-Error.JPG (202 KB) 03-Results-Error.JPG Phil Wardt, 03/31/2022 04:00 PM
nmap_scan-v18.patch (23.8 KB) nmap_scan-v18.patch Phil Wardt, 03/31/2022 04:03 PM
nmap_scan-v19.patch (23.8 KB) nmap_scan-v19.patch Phil Wardt, 04/01/2022 09:51 AM
nmap_scan-v20.patch (24 KB) nmap_scan-v20.patch Phil Wardt, 04/01/2022 04:21 PM
Start.JPG (205 KB) Start.JPG Phil Wardt, 04/01/2022 04:27 PM
nmap_scan-v21.patch (24 KB) nmap_scan-v21.patch Phil Wardt, 04/02/2022 04:02 AM
Actions
Copy link
 #1

Updated by Phil Wardt almost 3 years ago

Github commit, tested with screen shots:
https://github.com/pfsense/FreeBSD-ports/pull/1148

Note: it properly supports too big files like the packet capture command: will display a warning with path to the log file to download

Actions
Copy link
 #2

Updated by Phil Wardt almost 3 years ago

Add a working test patch that can be copied into System Patches package:

Actions
Copy link
 #3

Updated by Phil Wardt almost 3 years ago

Phil Wardt wrote in #note-2:

Add a working test patch that can be copied into System Patches package:

Added option to refresh/delete scan results log:
- if scan in progress: display refresh button
- if no scan results: no button is shown
- if scan results are present and no scan is running, offer clear scan results button

New patch file attached

Actions
Copy link
 #4

Updated by Phil Wardt almost 3 years ago

Phil Wardt wrote in #note-3:

Phil Wardt wrote in #note-2:

Add a working test patch that can be copied into System Patches package:

Added option to refresh/delete scan results log:
- if scan in progress: display refresh button
- if no scan results: no button is shown
- if scan results are present and no scan is running, offer clear scan results button

New patch file attached

A last one with a small optimisation: use php file_get_contents instead of system calls

Actions
Copy link
 #6

Updated by Jim Pingle almost 3 years ago

  • Tracker changed from Bug to Feature
  • Project changed from pfSense to pfSense Packages
  • Subject changed from Fix NMap timeout when started from GUI to Run nmap scans in the background
  • Category changed from Package System to Nmap
  • Assignee deleted (Viktor Gurov)
  • Release Notes deleted (Default)
Actions
Copy link
 #9

Updated by Phil Wardt almost 3 years ago

Updated TAB and Button names from ...log to "View Results"
Patch attached above

https://github.com/pfsense/FreeBSD-ports/pull/1148

Actions
Copy link
 #10

Updated by Jim Pingle over 2 years ago

  • Status changed from New to Pull Request Review
Actions
Copy link
 #11

Updated by Phil Wardt over 2 years ago

Standardize nmap text in description: NMap -> Nmap
https://github.com/pfsense/FreeBSD-ports/pull/1148

Actions
Copy link
 #12

Updated by Jim Pingle over 2 years ago

  • Status changed from Pull Request Review to Feedback
  • Assignee set to Jim Pingle

PR merged, thanks!

Actions
Copy link
 #13

Updated by Jim Pingle over 2 years ago

The Makefile needed an additional fix or it wouldn't compile: https://github.com/pfsense/FreeBSD-ports/commit/d34af18c95d6a45176d010648d1ad6991d8506a7

Actions
Copy link
 #14

Updated by Phil Wardt over 2 years ago

Again, noticed the delete icon resource
https://github.com/pfsense/FreeBSD-ports/pull/1152

Actions
Copy link
 #15

Updated by Phil Wardt over 2 years ago

After the last nmap changes, I wanted to harmonize the package with "Packet Capture"
https://github.com/pfsense/FreeBSD-ports/pull/1154

Source of nmap_scan.php followed packet capture php source and I made it a bit more structured
I also added a few features

Here are the log changes:

nmap command:
  • replace deprecated scan options (-sP)
  • use nmap built-in -oN summarized output format for results log
  • allow custom scan options
  • add support for IPsec and localhost interfaces
  • for -d option to no more allow deprecated options
GUI:
  • use same tab for all operations (harmonize gui with Packet Capture)
  • allow stop of running nmap processes started by gui
  • display error messages and command errors in a results error field
  • allow refresh results without stopping scan
  • default append to results log
  • properly summarized nmap log using -oN option
  • properly display the full command in output file using -oN option
  • hide detailed info in spannable tool tips

Images attached
Patch file attached: for testing, access with your pfsense.address/nmap_scan.php

Actions
Copy link
 #16

Updated by Phil Wardt over 2 years ago

To disable any code injection risks:
- input is matched against a white list allowing only alphanumeric, spaces (exclude tabs using specific 040 pattern), dots, - and _ chars
- "-d" option is forced on nmap to disable any deprecated options on version upgrades
- "-o" output command option is disabled
https://github.com/pfsense/FreeBSD-ports/blob/654af55a3208ea8c9aacf8f0a2618ab52f74206f/security/pfSense-pkg-nmap/files/usr/local/www/nmap_scan.php#L67

I am not sure if this is considered enough.
If not, I can disable the custom input fields and maybe just add a few useful scan methods and command options like traceroute. It will become much more useful than current implementation

Actions
Copy link
 #17

Updated by Jim Pingle over 2 years ago

  • Status changed from Feedback to Pull Request Review
Actions
Copy link
 #18

Updated by Phil Wardt over 2 years ago

I modified the code to disable any custom commands.
This is safer since nmap already changed in the past the -o option syntax and could do again in future.
I added a few more useful scan methods, ports syntax support, multiple hosts as per nmap syntax, and a few more options.
New screens attached

Patch file attached: for testing, access with url pfsense.address/nmap_scan.php

Actions
Copy link
 #19

Updated by Phil Wardt over 2 years ago

Updated patch to fix this:
- only kill nmap process using the output file created in GUI
- code formatting

Actions
Copy link
 #20

Updated by Phil Wardt over 2 years ago

Add No DNS Resolution option for faster scans
Should be completed

Attached patch for pfsense 2.6.0

Actions
Copy link
 #21

Updated by Phil Wardt over 2 years ago

I squashed commits since the last review
I reviewed and cleaned up some code readability
Updated the attached patch with the small code cleanup, no change in function

Actions
Copy link
 #22

Updated by Jim Pingle over 2 years ago

  • Status changed from Pull Request Review to Feedback

Merged to devel for testing in snapshots.

Actions
Copy link
 #23

Updated by Danilo Zrenjanin over 2 years ago

Tested the package against:

2.7.0-DEVELOPMENT (amd64)
built on Tue Apr 26 06:13:40 UTC 2022
FreeBSD 12.3-STABLE

The installation process went smoothly. I tested the package functionality and haven't noticed any issues. I'll leave it in Feedback status for one more round of testing.

Actions
Copy link
 #24

Updated by Marcos M over 2 years ago

Looks good from the testing I've done. Only suggestion I have is that the results file may be best placed in /tmp.

Actions
Copy link
 #25

Updated by Phil Wardt over 2 years ago

Marcos Mendoza wrote in #note-24:

Looks good from the testing I've done. Only suggestion I have is that the results file may be best placed in /tmp.

It was originally, like "Packet Capture" too, in /root. Not sure if it is for privacy ?

drwxr-xr-x   8 root  wheel    20 Apr 14 23:48 root
drwxrwxrwt   4 root  wheel    48 Jun  6 21:53 tmp

Let me know if you want me to push an amend with results put in /tmp

Actions
Copy link
 #26

Updated by Marcos M over 2 years ago

I can't think of a privacy issue for either - both locations are readable by everyone. The Packet Capture page is in need of review as well, but that's a separate issue.

Actions
Copy link
 #27

Updated by Phil Wardt over 2 years ago

Marcos M wrote in #note-26:

I can't think of a privacy issue for either - both locations are readable by everyone. The Packet Capture page is in need of review as well, but that's a separate issue.

Since it was merged in devel, should I commit changes to put results file in /tmp ? Or it is fine for now to be merged in other branches ?
I am not used to the commit way of pfsense
Since the original commit was merged in devel, the pull request was closed also

Actions
Copy link

Also available in: Atom PDF