Project

General

Profile

Bug #9211

GeoIP broken in pfSense-pkg-ntopng-0.8.13_3

Added by Mark Rosenstand over 1 year ago. Updated 2 months ago.

Status:
Pull Request Review
Priority:
Normal
Category:
ntop
Target version:
-
Start date:
12/18/2018
Due date:
% Done:

100%

Estimated time:
Affected Version:
2.4.4_1
Affected Architecture:

Description

Since upgrading ntopng (from previous to latest version), GeoIP support is broken. No flags are shown, listing by country doesn't work, and there are no traces on the Geo Map. Updating the GeoIP Data from the pfSense interface does not help (though I can see on the timestamp that the files in /usr/local/share/ntopng are indeed getting updated.)

It seems I'm not the only one with this problem: https://forum.netgate.com/topic/137688/ntopng-not-showing-country-flags

In the ntopng log, I get hundreds of messages "18/Dec/2018 23:04:00 [Mutex.cpp:46] WARNING: pthread_mutex_lock() returned 11 [Resource deadlock avoided][errno=0]" but it doesn't seem related.

ntopng.inc (11.2 KB) ntopng.inc YP Lo, 05/23/2019 07:30 AM
ntopng.inc.diff (4.56 KB) ntopng.inc.diff DRago_Angel [InV@DER], 08/16/2019 06:20 AM
ntopng.inc (11.2 KB) ntopng.inc Manjot Singh, 01/03/2020 07:42 PM

History

#1 Updated by Hijmen Schilperoort over 1 year ago

It is also not showing the locations (other than home location) and flows om the maps page

#2 Updated by Hijmen Schilperoort over 1 year ago

in the topic mentioned above there has been found a possible cause for this:

"Using pfSense 2.4.4-RELEASE-p2 with Ntopng Community Edition v.3.6.181116 I manually ran the /usr/local/bin/geoipupdate.sh script and got : fetch: http://geolite.maxmind.com/download/geoip/database/GeoLiteCountry/GeoIP.dat.gz: Not Found Then I went to Maxmind website and found this article: https://dev.maxmind.com/release-note/discontinuation-notice-for-geolite-legacy-databases/ It looks for me that Maxmind has ended their support and download for the legacy GeoIP data. The only GeoIP data you can download are the GeoLite2 data."

#3 Updated by YP Lo over 1 year ago

New ntopng 3.8 (December 2018) release supports the GeoIP2 library
  • Adds the new libmaxminddb geolocation library

#4 Updated by B D over 1 year ago

Any ETA on this please?

#5 Updated by B D over 1 year ago

PLEASE update the ntopng package -- surely this can't be that big of a deal!

#6 Updated by randombits b over 1 year ago

I have just installed Ntop only to find only parts of it are working - would it be possible to update this please ?

#7 Updated by B D about 1 year ago

It seems clear no one at Netgate is reading this ticket. :-(

#8 Updated by Tj Ng about 1 year ago

Looks like it :(. Anybody knows how to do a quick workaround and install 3.8 manually? or can I download the old version of the geoip databases?

I did try to replace the GeoIP database but that didn't work.

#9 Updated by Mark Vejvoda about 1 year ago

I got this working on my SG-3100 by copying files from:

https://centminmod.com/centminmodparts/geoip-legacy/

to a web server that I have control of, then I edited the urls in:
/usr/local/pkg/ntopng.inc
/usr/local/bin/geoipupdate.sh

Re-ran scripts manually from ssh to ensure there are no errors and clicked the delete followed by refresh geo button in the GUI ntopng settings,
restarted the service and now I am able to login and use ntopng using pfsense 2.4.4 p2 with ntopng 0.8.13_3

#10 Updated by Tj Ng about 1 year ago

Mark Vejvoda wrote:

I got this working on my SG-3100 by copying files from:

https://centminmod.com/centminmodparts/geoip-legacy/

to a web server that I have control of, then I edited the urls in:
/usr/local/pkg/ntopng.inc
/usr/local/bin/geoipupdate.sh

Re-ran scripts manually from ssh to ensure there are no errors and clicked the delete followed by refresh geo button in the GUI ntopng settings,
restarted the service and now I am able to login and use ntopng using pfsense 2.4.4 p2 with ntopng 0.8.13_3

So far not working for me.. Geo IP parts still not working:
  • No flags are shown
  • listing by country doesn't work
  • and there are no traces on the Geo Map.

wonder if I have broken some stuffs when I tried to install 3.8 manually. :/

#11 Updated by YP Lo about 1 year ago

Found out recently that ntopng v3.6 is already using GeoLite2 database, and hooked up the remaining GeoLite2 update scripts.

I have attached the updated ntopng.inc function so that GeoIP update appears to be working correctly on my end. Overwrite the file at /usr/local/pkg/ntopng.inc and run the GeoIP update via the GUI.

#12 Updated by Tj Ng about 1 year ago

YP Lo wrote:

Found out recently that ntopng v3.6 is already using GeoLite2 database, and hooked up the remaining GeoLite2 update scripts.

I have attached the updated ntopng.inc function so that GeoIP update appears to be working correctly on my end. Overwrite the file at /usr/local/pkg/ntopng.inc and run the GeoIP update via the GUI.

Nicely done! Flags are showing up now. GeoMap not working for me but I suspect it's missing the API key. Will look into that later.

Thanks :)

#13 Updated by Louis van Breda about 1 year ago

Hello,

I just installed and tested the package on 244 r3 and it is simply not working.

It realize that is possible to work around the issue with some tricks and downloads. But I would really appreciate if the bug was fixed (by generating a package based on v3.8 or later).

So I really regret that after six month there is still no action planned.

Sincerely,

Louis

#14 Updated by Tj Ng about 1 year ago

Louis van Breda wrote:

Hello,

I just installed and tested the package on 244 r3 and it is simply not working.

It realize that is possible to work around the issue with some tricks and downloads. But I would really appreciate if the bug was fixed (by generating a package based on v3.8 or later).

So I really regret that after six month there is still no action planned.

Was on 244_p2 and ntopng crashed after update to p3.

uninstalled ntopng, reinstalled and replaced with the new ntopng.inc provided earlier by Lo and it's working again.

As mentioned by Lo 3.6 already works with v2 Geo databases. So while the situation isn't ideal at least a workaround is available.

#18 Updated by Manjot Singh 6 months ago

As temporary solution I uploaded latest files on web server, and It seems to be working fine again.

#19 Updated by Manjot Singh 6 months ago

#21 Updated by Jim Pingle 3 months ago

  • Status changed from New to Pull Request Review

#22 Updated by Renato Botelho 2 months ago

  • Status changed from Pull Request Review to Feedback
  • Assignee set to Renato Botelho
  • % Done changed from 0 to 100

PR has been merged. Thanks!

#24 Updated by Jim Pingle 2 months ago

  • Status changed from Feedback to Pull Request Review

Also available in: Atom PDF