Project

General

Profile

Bug #9573

GeoIP database FAIL to download - Suricata package

Added by Carlos Montalvo J. about 2 months ago. Updated about 2 months ago.

Status:
New
Priority:
Normal-package
Category:
Suricata
Target version:
-
Start date:
06/02/2019
Due date:
% Done:

0%

Estimated time:
Affected Version:
2.4.4-p3
Affected Architecture:
amd64

Description

Hi, to everyone

Suricata v4.1.4 on pfSense 2.4.4-RELEASE-p3 (amd64)

Brand new suricata install, trying to get GeoIP Databases working by following installer prompts:
Message from GeoIP-1.6.12:

GeoIP does not ship with the actual data files. You must download
them yourself! To obtain the free database, run:
  1. /usr/local/bin/geoipupdate.sh

When the command is run in pfSense shell:
[2.4.4-RELEASE][root@pfsense]/root: /usr/local/bin/geoipupdate.sh
Fetching GeoIP.dat and GeoIPv6.dat...
fetch: http://geolite.maxmind.com/download/geoip/database/GeoLiteCountry/GeoIP.dat.gz: Not Found
GeoIP.dat download failed

Upstream obsoleted download function:
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=234715

Possible solution: -- Replace download path to:
https://centminmod.com/centminmodparts/geoip-legacy/
(Solution taken from: https://redmine.pfsense.org/issues/9211/quoted?journal_id=40451)

History

#1 Updated by Bill Meeks about 2 months ago

You do not need to do anything to use the free GeoIP2 Lite database with Suricata on pfSense. It is automatically set up at package installation and a daily update cron job is created to keep the database updated from that point forward. This assumes you are using the free version of the GeoIP2 Lite database.

The database you are trying to download has been deprecated by MaxMind. GeoIP has been replaced by GeoIP2. The new database has a completely new internal format and is incompatible with applications that used the old GeoIP format. Suricata was updated for the 4.1.2 release to use the GeoIP2 database.

If you have a paid GeoIP2 database subscription and corresponding code from MaxMind, then post back and I can post some detailed instructions on how to hanlde that with the Suricata package. It will take manual editing of some files. Otherwise, remove the package you installed that provided the /usr/local/bin/geoipupdate.sh script. It is not needed for Suricata 4.1.4 on pfSense.

#2 Updated by Carlos Montalvo J. about 2 months ago

Hi, Bill

I´m sorry but suricata is the one installing package GeoIP-1.6.12.

Only the following packages are installed on my pfSense:
  • Avahi v2.0.0_2
  • Filer v0.60.6_1
  • lldpd v0.9.9
  • pfBlockerNG v2.1.4_17
  • Service_Watchdog v1.8.6
  • Suricata v4.1.4

Here is a copy of a the install output following the WebGUI, after I removed Suricata and proceed to reinstall. (Also suricata should display all required dependencies not just barnyard2)

Installing pfSense-pkg-suricata...

Updating pfSense-core repository catalogue...
pfSense-core repository is up to date.
Updating pfSense repository catalogue...
pfSense repository is up to date.
All repositories are up to date.
Checking integrity... done (0 conflicting)
The following 14 package(s) will be affected (of 0 checked):

New packages to be INSTALLED:
pfSense-pkg-suricata: 4.1.4 [pfSense]
suricata: 4.1.4 [pfSense]
libyaml: 0.1.6_2 [pfSense]
nss: 3.39 [pfSense]
nspr: 4.20 [pfSense]
libpcap: 1.8.1 [pfSense]
libnet: 1.1.6_5,1 [pfSense]
py27-yaml: 5.1 [pfSense]
hyperscan: 4.6.0 [pfSense]
hiredis: 0.13.3 [pfSense]
barnyard2: 1.13_1 [pfSense]
broccoli: 1.97,1 [pfSense]
*GeoIP: 1.6.12 [pfSense]*++
mysql56-client: 5.6.41 [pfSense]

Number of packages to be installed: 14

The process will require 76 MiB more space.
[1/14] Installing nspr-4.20...
[1/14] Extracting nspr-4.20: .......... done
[2/14] Installing GeoIP-1.6.12...
[2/14] Extracting GeoIP-1.6.12: .......... done
[3/14] Installing libyaml-0.1.6_2...
[3/14] Extracting libyaml-0.1.6_2: ......... done
[4/14] Installing nss-3.39...
[4/14] Extracting nss-3.39: .......... done
[5/14] Installing libpcap-1.8.1...
[5/14] Extracting libpcap-1.8.1: .......... done
[6/14] Installing libnet-1.1.6_5,1...
[6/14] Extracting libnet-1.1.6_5,1: .......... done
[7/14] Installing py27-yaml-5.1...
[7/14] Extracting py27-yaml-5.1: .......... done
[8/14] Installing hyperscan-4.6.0...
[8/14] Extracting hyperscan-4.6.0: .......... done
[9/14] Installing hiredis-0.13.3...
[9/14] Extracting hiredis-0.13.3: .......... done
[10/14] Installing broccoli-1.97,1...
[10/14] Extracting broccoli-1.97,1: .......... done
[11/14] Installing mysql56-client-5.6.41...
[11/14] Extracting mysql56-client-5.6.41: .......... done
[12/14] Installing suricata-4.1.4...
[12/14] Extracting suricata-4.1.4: .......... done
[13/14] Installing barnyard2-1.13_1...
[13/14] Extracting barnyard2-1.13_1: ...... done
[14/14] Installing pfSense-pkg-suricata-4.1.4...
[14/14] Extracting pfSense-pkg-suricata-4.1.4: .......... done
Saving updated package information...
done.
Loading package configuration... done.
Configuring package components...
Loading package instructions...
Custom commands...
Executing custom_php_install_command()...Saved settings detected...
Migrating settings to new configuration... done.
Downloading Emerging Threats Open rules md5 file... done.
There is a new set of Emerging Threats Open rules posted. Downloading... done.
Downloading Snort GPLv2 Community Rules md5 file... done.
There is a new set of Snort GPLv2 Community Rules posted. Downloading... done.
Installing Emerging Threats Open rules... done.
Installing Snort GPLv2 Community Rules... done.
Updating rules configuration for: WAN ... done.
Cleaning up after rules extraction... done.
The Rules update has finished.
Generating suricata.yaml configuration file from saved settings.
Generating YAML configuration file for WAN... done.
Finished rebuilding Suricata configuration from saved settings.
Setting package version in configuration file.
done.
Executing custom_php_resync_config_command()...done.
Menu items... done.
Services... done.
Writing configuration... done.
Message from GeoIP-1.6.12:

GeoIP does not ship with the actual data files. You must download
them yourself! To obtain the free database, run:
  1. /usr/local/bin/geoipupdate.sh
    Message from mysql56-client-5.6.41:
  • * * * * * * * * * * * * * * * * * * * * * * *

Please be aware the database client is vulnerable
to CVE-2015-3152 - SSL Downgrade aka "BACKRONYM".
You may find more information at the following URL:

http://www.vuxml.org/freebsd/36bd352d-299b-11e5-86ff-14dae9d210b8.html

Although this database client is not listed as
"affected", it is vulnerable and will not be
receiving a patch. Please take note of this when
deploying this software.

  • * * * * * * * * * * * * * * * * * * * * * * *
    Message from suricata-4.1.4:

===========================================================================

If you want to run Suricata in IDS mode, add to /etc/rc.conf:

suricata_enable="YES" 
suricata_interface="<if>"

NOTE: Declaring suricata_interface is MANDATORY for Suricata in IDS Mode.

However, if you want to run Suricata in Inline IPS Mode in divert(4) mode,
add to /etc/rc.conf:

suricata_enable="YES" 
suricata_divertport="8000"

NOTE:
Suricata won't start in IDS mode without an interface configured.
Therefore if you omit suricata_interface from rc.conf, FreeBSD's
rc.d/suricata will automatically try to start Suricata in IPS Mode
(on divert port 8000, by default).

Alternatively, if you want to run Suricata in Inline IPS Mode in high-speed
netmap(4) mode, add to /etc/rc.conf:

suricata_enable="YES" 
suricata_netmap="YES"

NOTE:
Suricata requires additional interface settings in the configuration
file to run in netmap(4) mode.

RULES: Suricata IDS/IPS Engine comes without rules by default. You should
add rules by yourself and set an updating strategy. To do so, please visit:

http://www.openinfosecfoundation.org/documentation/rules.html
http://www.openinfosecfoundation.org/documentation/emerging-threats.html

You may want to try BPF in zerocopy mode to test performance improvements:

sysctl -w net.bpf.zerocopy_enable=1

Don't forget to add net.bpf.zerocopy_enable=1 to /etc/sysctl.conf

===========================================================================
Message from barnyard2-1.13_1:

Read the notes in the barnyard2.conf file for how to configure
/usr/local/etc/barnyard2.conf after installation. For addtional information
see the Securixlive FAQ at http://www.securixlive.com/barnyard2/faq.php.

In order to enable barnyard2 to start on boot, you must edit /etc/rc.conf
with the appropriate flags, etc. See the FreeBSD Handbook for syntax:
http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/configtuning-rcng.html

For the various options available, type % barnyard2 -h after install or read
the options in the startup script - in /usr/local/etc/rc.d.

Barnyard2 can process unified2 files from snort or suricata. It can also
interact with snortsam firewall rules as well as the sguil-sensor. Those
ports must be installed separately if you wish to use them.


Cleaning up cache... done.

Success

--
Bill Meeks wrote:

You do not need to do anything to use the free GeoIP2 Lite database with Suricata on pfSense. It is automatically set up at package installation and a daily update cron job is created to keep the database updated from that point forward. This assumes you are using the free version of the GeoIP2 Lite database.

The database you are trying to download has been deprecated by MaxMind. GeoIP has been replaced by GeoIP2. The new database has a completely new internal format and is incompatible with applications that used the old GeoIP format. Suricata was updated for the 4.1.2 release to use the GeoIP2 database.

If you have a paid GeoIP2 database subscription and corresponding code from MaxMind, then post back and I can post some detailed instructions on how to hanlde that with the Suricata package. It will take manual editing of some files. Otherwise, remove the package you installed that provided the /usr/local/bin/geoipupdate.sh script. It is not needed for Suricata 4.1.4 on pfSense.

#3 Updated by Bill Meeks about 2 months ago

Hmm... looks like it is getting pulled in as a dependency, probably with a library.

No matter, you still do not use it as it won't be properly configured. The pfSense installation of Suricata will trigger an internal Suricata script that will download the proper database and place it in /usr/local/share/suricata/GeoLite2/. The filename should be GeoLite2-Country.mmdb and it should show a recent date. The free database is updated once per month, I believe, so the date could be one month old or slightly more.

The file that installs the database and then keeps it updated is /usr/local/pkg/suricata/suricata_geoipupdate.php. That file is run once during package installation and then is set up as a daily cron task.

That message you are seeing is the default post-install messages that are part of the upstream FreeBSD port. They really have no relevance when you run one of the pfSense GUI packages like Suricata or Snort.

Also available in: Atom PDF