Project

General

Profile

Actions
Copy link

Bug #9223

closed

SSHGUARD doesn't work as expected

Added by Joshua Sign almost 6 years ago. Updated over 5 years ago.

Status:
Resolved
Priority:
High
Assignee:
Renato Botelho
Category:
Rules / NAT
Target version:
2.4.4-p3
Start date:
12/26/2018
Due date:
% Done:

100%

Estimated time:
Plus Target Version:
Release Notes:
Affected Version:
2.4.x
Affected Architecture:
All

Description

Sshguard implementation in pfsense broke the way that sshguard should work.

I notice that blocking IP for a while (many hours) is not possible because of crontab tasks :

*/60   *       *       *       *       root    /usr/bin/nice -n20 /usr/local/sbin/expiretable -v -t 3600 sshguard
*/60   *       *       *       *       root    /usr/bin/nice -n20 /usr/local/sbin/expiretable -v -t 3600 webConfiguratorlockout

According to the manual, "expiretable will remove entries from the pf table specified by table with an age greater than that specified by -t age".
So all entries olders than 1h will be deleted : this is the first problem, because sshguard do not realease himself these entries it will not block again these IP since the interval time is not reached.

As sshguard do not block them again, after the cron job, many logs lines like this one appears in system logs (about IP that should ever be bloked) :

Dec 26 10:50:17    sshd    13972    Unable to negotiate with 218.92.1.172 port 10899: no matching key exchange method found. Their offer: diffie-hellman-group14-sha1,diffie-hellman-group-exchange-sha1,diffie-hellman-group1-sha1 [preauth]
Dec 26 10:50:13    sshd    47006   Unable to negotiate with 218.92.1.172 port 56425: no matching key exchange method found. Their offer: diffie-hellman-group14-sha1,diffie-hellman-group-exchange-sha1,diffie-hellman-group1-sha1 [preauth]
Dec 26 10:50:10    sshd    70133   Unable to negotiate with 218.92.1.172 port 63321: no matching key exchange method found. Their offer: diffie-hellman-group14-sha1,diffie-hellman-group-exchange-sha1,diffie-hellman-group1-sha1 [preauth]

That makes logs grow and grow...

Sshguard works correctly using backends to block/release IP.
These cron jobs are not required and broke the sshguard interval timings.

But, the main error is certainly this unexpected behavior : if sshguard block your IP for a services, it will not block it again for another service.
Just try to connect to https with a bad login/password :

Dec 26 10:55:45    sshguard    55911    Blocking "10.0.0.10/32" for 2400 secs (1 attacks in 0 secs, after 3 abuses over 34151 secs.)
Dec 26 10:55:45    sshguard    55911    Attack from "10.0.0.10" on service 380 with danger 10.
Dec 26 10:55:45    php-fpm     20430    /index.php: webConfigurator authentication error for user 'toto' from: 10.0.0.10

Now we are blocked over https... ok but what about ssh ?
Lets try :

Dec 26 10:57:24    sshd    82837    Disconnecting invalid user titi 10.0.0.10 port 51346: Too many authentication failures [preauth]
Dec 26 10:57:24    sshd    82837    error: maximum authentication attempts exceeded for invalid user titi from 10.0.0.10 port 51346 ssh2 [preauth]
Dec 26 10:57:24    sshd    82837    Failed keyboard-interactive/pam for invalid user titi from 10.0.0.10 port 51346 ssh2
Dec 26 10:57:24    sshd    82837    error: PAM: authentication error for illegal user titi from 10.0.0.10
Dec 26 10:57:22    sshd    82837    Postponed keyboard-interactive for invalid user titi from 10.0.0.10 port 51346 ssh2 [preauth]
Dec 26 10:57:22    sshd    82837    user NOUSER login class [preauth]
Dec 26 10:57:22    sshd    82837    Failed keyboard-interactive/pam for invalid user titi from 10.0.0.10 port 51346 ssh2
Dec 26 10:57:22    sshd    82837    error: PAM: authentication error for illegal user titi from 10.0.0.10
Dec 26 10:57:21    sshd    82837    Postponed keyboard-interactive for invalid user titi from 10.0.0.10 port 51346 ssh2 [preauth]
Dec 26 10:57:21    sshd    82837    user NOUSER login class [preauth]
Dec 26 10:57:21    sshd    82837    Failed keyboard-interactive/pam for invalid user titi from 10.0.0.10 port 51346 ssh2
Dec 26 10:57:21    sshd    82837    error: PAM: authentication error for illegal user titi from 10.0.0.10
Dec 26 10:57:21    sshd    82837    Postponed keyboard-interactive for invalid user titi from 10.0.0.10 port 51346 ssh2 [preauth]
Dec 26 10:57:21    sshd    82837    user NOUSER login class [preauth]
Dec 26 10:57:21    sshd    82837    Failed keyboard-interactive/pam for invalid user titi from 10.0.0.10 port 51346 ssh2
Dec 26 10:57:21    sshd    82837    error: PAM: authentication error for illegal user titi from 10.0.0.10
Dec 26 10:57:20    sshd    82837    Postponed keyboard-interactive for invalid user titi from 10.0.0.10 port 51346 ssh2 [preauth]
Dec 26 10:57:20    sshd    82837    user NOUSER login class [preauth]
Dec 26 10:57:20    sshd    82837    Failed keyboard-interactive/pam for invalid user titi from 10.0.0.10 port 51346 ssh2
Dec 26 10:57:20    sshd    82837    error: PAM: authentication error for illegal user titi from 10.0.0.10
Dec 26 10:57:19    sshd    82837    Postponed keyboard-interactive for invalid user titi from 10.0.0.10 port 51346 ssh2 [preauth]
Dec 26 10:57:19    sshd    82837    user NOUSER login class [preauth]
Dec 26 10:57:19    sshd    82837    Failed keyboard-interactive/pam for invalid user titi from 10.0.0.10 port 51346 ssh2
Dec 26 10:57:19    sshd    82837    error: PAM: authentication error for illegal user titi from 10.0.0.10
Dec 26 10:57:18    sshd    82837    Postponed keyboard-interactive for invalid user titi from 10.0.0.10 port 51346 ssh2 [preauth]
Dec 26 10:57:18    sshd    82837    user NOUSER login class [preauth]
Dec 26 10:57:18    sshd    82837    user NOUSER login class [preauth]
Dec 26 10:57:18    sshd    82837    Invalid user titi from 10.0.0.10 port 51346
Dec 26 10:57:12    sshd    39027    Disconnecting invalid user test 10.0.0.10 port 51327: Too many authentication failures [preauth]
Dec 26 10:57:12    sshd    39027    error: maximum authentication attempts exceeded for invalid user test from 10.0.0.10 port 51327 ssh2 [preauth]
Dec 26 10:57:12    sshd    39027    Failed keyboard-interactive/pam for invalid user test from 10.0.0.10 port 51327 ssh2
Dec 26 10:57:12    sshd    39027    error: PAM: authentication error for illegal user test from 10.0.0.10
Dec 26 10:57:11    sshd    39027    Postponed keyboard-interactive for invalid user test from 10.0.0.10 port 51327 ssh2 [preauth]
Dec 26 10:57:11    sshd    39027    user NOUSER login class [preauth]
Dec 26 10:57:11    sshd    39027    Failed keyboard-interactive/pam for invalid user test from 10.0.0.10 port 51327 ssh2
Dec 26 10:57:11    sshd    39027    error: PAM: authentication error for illegal user test from 10.0.0.10
Dec 26 10:57:10    sshd    39027    Postponed keyboard-interactive for invalid user test from 10.0.0.10 port 51327 ssh2 [preauth]
Dec 26 10:57:10    sshd    39027    user NOUSER login class [preauth]
Dec 26 10:57:10    sshd    39027    Failed keyboard-interactive/pam for invalid user test from 10.0.0.10 port 51327 ssh2
Dec 26 10:57:10    sshd    39027    error: PAM: authentication error for illegal user test from 10.0.0.10
Dec 26 10:57:09    sshd    39027    Postponed keyboard-interactive for invalid user test from 10.0.0.10 port 51327 ssh2 [preauth]
Dec 26 10:57:09    sshd    39027    user NOUSER login class [preauth]
Dec 26 10:57:09    sshd    39027    Failed keyboard-interactive/pam for invalid user test from 10.0.0.10 port 51327 ssh2
Dec 26 10:57:09    sshd    39027    error: PAM: authentication error for illegal user test from 10.0.0.10
Dec 26 10:57:08    sshd    39027    Postponed keyboard-interactive for invalid user test from 10.0.0.10 port 51327 ssh2 [preauth]
Dec 26 10:57:08    sshd    39027    user NOUSER login class [preauth]
Dec 26 10:57:08    sshd    39027    Failed keyboard-interactive/pam for invalid user test from 10.0.0.10 port 51327 ssh2
Dec 26 10:57:08    sshd    39027    error: PAM: authentication error for illegal user test from 10.0.0.10
Dec 26 10:57:06    sshd    39027    Postponed keyboard-interactive for invalid user test from 10.0.0.10 port 51327 ssh2 [preauth]
Dec 26 10:57:06    sshd    39027    user NOUSER login class [preauth]
Dec 26 10:57:06    sshd    39027    Failed keyboard-interactive/pam for invalid user test from 10.0.0.10 port 51327 ssh2
Dec 26 10:57:06    sshd    39027    error: PAM: authentication error for illegal user test from 10.0.0.10
Dec 26 10:57:05    sshd    39027    Postponed keyboard-interactive for invalid user test from 10.0.0.10 port 51327 ssh2 [preauth]
Dec 26 10:57:05    sshd    39027    user NOUSER login class [preauth]
Dec 26 10:57:05    sshd    39027    user NOUSER login class [preauth]
Dec 26 10:57:05    sshd    39027    Invalid user test from 10.0.0.10 port 51327
Dec 26 10:56:15    sshd    12848    Fssh_packet_write_poll: Connection from user root 10.0.0.10 port 49560: Permission denied

Ssh is not blocked!!

I show many attacks like that, so it is possible that some hackers understand how to bypass sshguard protection using this trick to bruteforce.

For now, and as a minimal workaround, i suggest to remove or comments crontabs lines : that solve many problems and make sshguard works as expected

I need to investigate more about the service blocking bypass, but this is a real problem.
During this time, maybe it should be more secure to block both services at once...

I'am using :

2.4.4-RELEASE-p1 (amd64) 
built on Mon Nov 26 11:40:26 EST 2018 
FreeBSD 11.2-RELEASE-p4

Files

sshguad_by_service_filtering.patch (7.91 KB) sshguad_by_service_filtering.patch Joshua Sign, 01/20/2019 04:32 PM
Actions
Copy link
 #1

Updated by Jim Pingle almost 6 years ago

  • Category set to Rules / NAT
  • Assignee set to Renato Botelho
  • Target version set to 48
  • Affected Version set to 2.4.4_1
Actions
Copy link
 #2

Updated by Danilo Zrenjanin almost 6 years ago

I have reproduced the bug on SG-3100:

2.4.4-RELEASE-p1 (arm)
built on Thu Nov 29 14:06:34 EST 2018
FreeBSD 11.2-RELEASE-p4

I noticed if you have been locked over failed HTTPS attempts you will be able to continue with SSH attempts. But if you've been blocked due to failed SSH attempts you will be blocked over HTTPS too.

Same results on SG-3100:

2.4.5-DEVELOPMENT (arm)
built on Thu Dec 27 19:12:05 EST 2018
FreeBSD 11.2-RELEASE-p6

Actions
Copy link
 #3

Updated by Joshua Sign almost 6 years ago

Hi Danilo,

I'am not agree with your test.

I just test again to be sure about it, and i can confirm that if your are blocked due to failed SSH attempts, you ARE NOT BLOCKED over HTTPS.
Here is my test :

Dec 29 12:16:43    php-fpm       474    /index.php: webConfigurator authentication error for user 'tutu' from: 192.168.100.10
Dec 29 12:16:39    php-fpm     32814    /index.php: webConfigurator authentication error for user 'tata' from: 192.168.100.10
Dec 29 12:16:24    php-fpm     49514    /index.php: webConfigurator authentication error for user 'titi' from: 192.168.100.10
Dec 29 12:16:06    php-fpm     24638    /index.php: webConfigurator authentication error for user 'toto' from: 192.168.100.10
Dec 29 12:15:26    sshguard    88267    Blocking "192.168.100.10/32" for 600 secs (1 attacks in 0 secs, after 1 abuses over 0 secs.)
Dec 29 12:15:26    sshguard    88267    Attack from "192.168.100.10" on service 100 with danger 10.
Dec 29 12:15:26    sshd        46068    Invalid user test from 192.168.100.10 port 60692

At 12:15:26 i do ssh connexion failled and i'am blocked by sshguard.
But i can try many attempts over HTTPS...

I check tables too :
- sshguard table list got my ip address inside : good because i have failed an attempt over SSH
- webConfiguratorlockout table list dont have my ip listed inside -> my IP should be added after the first HTTPS connexion failed, but sshguard don't add/detect it

I can perform more tests if needed.

Regards.
Josh_

Actions
Copy link
 #4

Updated by Danilo Zrenjanin almost 6 years ago

Retested and got the same results as Joshua. I must have messed up something with IPs or Safari browser got stuck during the first test.

Correct, if you are blocked due to failed SSH attempts, you ARE NOT BLOCKED over HTTPS.

Actions
Copy link
 #5

Updated by Joshua Sign almost 6 years ago

I investigate about this problem,

It seems that the sshguard purpose is to detect an attack and just launch a backend script to command firewall and block the attacker.
The attacker is supposed to be blocked for all services, in sshguard purpose.

As it could be very interresting to have sshguard blocking ip by services, i just worked on it.
Here is the patch. My tests looks good on debian system.

With specific backend rules we will now be able to block only on HTTPS, and/or on SSH, and/or on openvpn, and/or ...

This patch :
- add the service number to the backend script arguments
- make attackers blocking/unblocking by services
- message log when unblocking (before it was LOG_DEBUG only)

Actions
Copy link
 #7

Updated by Jim Pingle almost 6 years ago

  • Target version changed from 48 to 2.5.0
Actions
Copy link
 #8

Updated by Jim Pingle almost 6 years ago

  • % Done changed from 0 to 10

I pushed a change to remove the cron job. Additional changes are coming shortly.

Actions
Copy link
 #9

Updated by Jim Pingle almost 6 years ago

  • Status changed from New to Feedback
  • % Done changed from 10 to 100
  • Affected Version changed from 2.4.4_1 to 2.4.x
  • Affected Architecture All added
  • Affected Architecture deleted ()

sshguard 2.3.1 is now present on 2.5.0 snapshots being tested. It has the extra GUI table code removed.

Associated code changes have been committed to use only the sshguard table as well for both ssh and GUI lockouts.

Passes all internal testing so far, but can't be tested by others until we have public 2.5.0 snapshots.

So in all:
  • sshguard changed to only use a single table, sshguard.
  • expiretable cron jobs for sshguard-managed tables removed from the configuration on next upgrade
  • expiretable cron jobs removed from the default config.xml
  • webConfiguratorlockout table removed, now only sshguard table is used
Actions
Copy link
 #10

Updated by Jim Pingle almost 6 years ago

Joshua Sign wrote:

As it could be very interresting to have sshguard blocking ip by services, i just worked on it.
Here is the patch. My tests looks good on debian system.

We opted not to add any more patches on top of sshguard, but you should absolutely submit that upstream to the sshguard project. They have been receptive to changes, and if others would benefit from the feature, then getting it in upstream is the best way to make that happen.

Actions
Copy link
 #11

Updated by Jim Pingle over 5 years ago

  • Parent task set to #9398
Actions
Copy link
 #12

Updated by Jim Pingle over 5 years ago

  • Target version changed from 2.5.0 to 2.4.4-p3
Actions
Copy link
 #13

Updated by Jim Pingle over 5 years ago

  • Parent task changed from #9398 to #9515
Actions
Copy link
 #14

Updated by Joshua Sign over 5 years ago

Jim Pingle wrote:

We opted not to add any more patches on top of sshguard, but you should absolutely submit that upstream to the sshguard project. They have been receptive to changes, and if others would benefit from the feature, then getting it in upstream is the best way to make that happen.

Hi Jim,

I ever submit to the sshguard team, but they move slowly.
https://bitbucket.org/sshguard/sshguard/pull-requests/46/add-pfsense-signature/diff

The branche i use for now is mine : https://bitbucket.org/JoshuaSign/sshguard/branch/sshguard_featured
And it works like a charm for me.

But because this code is my first one in C langage, i cant be sure it is safe.
An expert should take a look a t it before use.

Do the best you can do.
Regards.

Actions
Copy link
 #15

Updated by Steve Wheeler over 5 years ago

  • Status changed from Feedback to Resolved

Confirmed against CE 2.4.4p2. Triggering lockout via SSH still allows unlimited login attempts at the gui.

Confirmed that does not happen on CE 2.4.4p3. Triggering attacks from either will lockout both.

Actions
Copy link

Also available in: Atom PDF