Project

General

Profile

Actions

Bug #9211

closed

GeoIP broken in pfSense-pkg-ntopng-0.8.13_3

Added by Mark Rosenstand almost 3 years ago. Updated about 1 year ago.

Status:
Resolved
Priority:
Normal
Category:
ntop
Target version:
-
Start date:
12/18/2018
Due date:
% Done:

100%

Estimated time:
Plus Target Version:
Affected Version:
2.4.4_1
Affected Plus Version:
Affected Architecture:

Description

Since upgrading ntopng (from previous to latest version), GeoIP support is broken. No flags are shown, listing by country doesn't work, and there are no traces on the Geo Map. Updating the GeoIP Data from the pfSense interface does not help (though I can see on the timestamp that the files in /usr/local/share/ntopng are indeed getting updated.)

It seems I'm not the only one with this problem: https://forum.netgate.com/topic/137688/ntopng-not-showing-country-flags

In the ntopng log, I get hundreds of messages "18/Dec/2018 23:04:00 [Mutex.cpp:46] WARNING: pthread_mutex_lock() returned 11 [Resource deadlock avoided][errno=0]" but it doesn't seem related.


Files

ntopng.inc (11.2 KB) ntopng.inc YP Lo, 05/23/2019 07:30 AM
ntopng.inc.diff (4.56 KB) ntopng.inc.diff DRago_Angel [InV@DER], 08/16/2019 06:20 AM
ntopng.inc (11.2 KB) ntopng.inc Manjot Singh, 01/03/2020 07:42 PM
Actions #1

Updated by Hijmen Schilperoort almost 3 years ago

It is also not showing the locations (other than home location) and flows om the maps page

Actions #2

Updated by Hijmen Schilperoort almost 3 years ago

in the topic mentioned above there has been found a possible cause for this:

"Using pfSense 2.4.4-RELEASE-p2 with Ntopng Community Edition v.3.6.181116 I manually ran the /usr/local/bin/geoipupdate.sh script and got : fetch: http://geolite.maxmind.com/download/geoip/database/GeoLiteCountry/GeoIP.dat.gz: Not Found Then I went to Maxmind website and found this article: https://dev.maxmind.com/release-note/discontinuation-notice-for-geolite-legacy-databases/ It looks for me that Maxmind has ended their support and download for the legacy GeoIP data. The only GeoIP data you can download are the GeoLite2 data."

Actions #3

Updated by YP Lo over 2 years ago

New ntopng 3.8 (December 2018) release supports the GeoIP2 library
  • Adds the new libmaxminddb geolocation library
Actions #4

Updated by B D over 2 years ago

Any ETA on this please?

Actions #5

Updated by B D over 2 years ago

PLEASE update the ntopng package -- surely this can't be that big of a deal!

Actions #6

Updated by randombits b over 2 years ago

I have just installed Ntop only to find only parts of it are working - would it be possible to update this please ?

Actions #7

Updated by B D over 2 years ago

It seems clear no one at Netgate is reading this ticket. :-(

Actions #8

Updated by Tj Ng over 2 years ago

Looks like it :(. Anybody knows how to do a quick workaround and install 3.8 manually? or can I download the old version of the geoip databases?

I did try to replace the GeoIP database but that didn't work.

Actions #9

Updated by Mark Vejvoda over 2 years ago

I got this working on my SG-3100 by copying files from:

https://centminmod.com/centminmodparts/geoip-legacy/

to a web server that I have control of, then I edited the urls in:
/usr/local/pkg/ntopng.inc
/usr/local/bin/geoipupdate.sh

Re-ran scripts manually from ssh to ensure there are no errors and clicked the delete followed by refresh geo button in the GUI ntopng settings,
restarted the service and now I am able to login and use ntopng using pfsense 2.4.4 p2 with ntopng 0.8.13_3

Actions #10

Updated by Tj Ng over 2 years ago

Mark Vejvoda wrote:

I got this working on my SG-3100 by copying files from:

https://centminmod.com/centminmodparts/geoip-legacy/

to a web server that I have control of, then I edited the urls in:
/usr/local/pkg/ntopng.inc
/usr/local/bin/geoipupdate.sh

Re-ran scripts manually from ssh to ensure there are no errors and clicked the delete followed by refresh geo button in the GUI ntopng settings,
restarted the service and now I am able to login and use ntopng using pfsense 2.4.4 p2 with ntopng 0.8.13_3

So far not working for me.. Geo IP parts still not working:
  • No flags are shown
  • listing by country doesn't work
  • and there are no traces on the Geo Map.

wonder if I have broken some stuffs when I tried to install 3.8 manually. :/

Actions #11

Updated by YP Lo over 2 years ago

Found out recently that ntopng v3.6 is already using GeoLite2 database, and hooked up the remaining GeoLite2 update scripts.

I have attached the updated ntopng.inc function so that GeoIP update appears to be working correctly on my end. Overwrite the file at /usr/local/pkg/ntopng.inc and run the GeoIP update via the GUI.

Actions #12

Updated by Tj Ng over 2 years ago

YP Lo wrote:

Found out recently that ntopng v3.6 is already using GeoLite2 database, and hooked up the remaining GeoLite2 update scripts.

I have attached the updated ntopng.inc function so that GeoIP update appears to be working correctly on my end. Overwrite the file at /usr/local/pkg/ntopng.inc and run the GeoIP update via the GUI.

Nicely done! Flags are showing up now. GeoMap not working for me but I suspect it's missing the API key. Will look into that later.

Thanks :)

Actions #13

Updated by Louis van Breda over 2 years ago

Hello,

I just installed and tested the package on 244 r3 and it is simply not working.

It realize that is possible to work around the issue with some tricks and downloads. But I would really appreciate if the bug was fixed (by generating a package based on v3.8 or later).

So I really regret that after six month there is still no action planned.

Sincerely,

Louis

Actions #14

Updated by Tj Ng over 2 years ago

Louis van Breda wrote:

Hello,

I just installed and tested the package on 244 r3 and it is simply not working.

It realize that is possible to work around the issue with some tricks and downloads. But I would really appreciate if the bug was fixed (by generating a package based on v3.8 or later).

So I really regret that after six month there is still no action planned.

Was on 244_p2 and ntopng crashed after update to p3.

uninstalled ntopng, reinstalled and replaced with the new ntopng.inc provided earlier by Lo and it's working again.

As mentioned by Lo 3.6 already works with v2 Geo databases. So while the situation isn't ideal at least a workaround is available.

Actions #18

Updated by Manjot Singh almost 2 years ago

As temporary solution I uploaded latest files on web server, and It seems to be working fine again.

Actions #19

Updated by Manjot Singh almost 2 years ago

Actions #21

Updated by Jim Pingle over 1 year ago

  • Status changed from New to Pull Request Review
Actions #22

Updated by Renato Botelho over 1 year ago

  • Status changed from Pull Request Review to Feedback
  • Assignee set to Renato Botelho
  • % Done changed from 0 to 100

PR has been merged. Thanks!

Actions #24

Updated by Jim Pingle over 1 year ago

  • Status changed from Feedback to Pull Request Review
Actions #25

Updated by Renato Botelho about 1 year ago

  • Status changed from Pull Request Review to Feedback

PR has been merged

Actions #26

Updated by Danilo Zrenjanin about 1 year ago

  • Status changed from Feedback to Resolved

Tested on :

2.5.0-DEVELOPMENT (amd64)
built on Fri Oct 16 19:03:23 EDT 2020
FreeBSD 12.2-STABLE

GeoIP works as expected. Ticket resolved.

Actions

Also available in: Atom PDF