Project

General

Profile

Actions

Bug #14577

closed

OpenVPN not removing old Cisco-AVPair anchor rules and files in ``/tmp``

Added by Michael Mercier 10 months ago. Updated 7 months ago.

Status:
Needs Patch
Priority:
Normal
Assignee:
Category:
OpenVPN
Target version:
Start date:
Due date:
% Done:

0%

Estimated time:
Plus Target Version:
Plus-Next
Release Notes:
Default
Affected Version:
Affected Architecture:
6100

Description

Hello,

I am seeing duplicate ovpn_ovpns1_<user>_<port>.rules files in the /tmp directory, and I also see duplicate entries when running pfSsh playback pfanchordrill

Another observation I have made:

Some users have a /tmp/<user> file containing routes, some don't.

As an example:

[23.05.1-RELEASE][root@vpn]/tmp:ls ovpn_ovpns1_* | awk -F'_' '{print $3}' | xargs ls
ls: user105: No such file or directory
ls: user109: No such file or directory
ls: user110: No such file or directory
ls: user115: No such file or directory
ls: user119: No such file or directory
user100    user106    user116
user101    user107    user117
user101    user108    user118
user102    user111    user120
user103    user112    user121
user104    user113    user121
user104    user115    user122

NOTE:
The 6100 was upgraded to 23.05.1 yesterday (July 12th, 2023). Before that the device was running 22.05, I was experiencing https://redmine.pfsense.org/issues/12332 on the device.

Please let me know if there is any additional information I can provide.

Thanks,
Mike


Related issues

Related to Bug #12332: OpenVPN does not clear old Cisco-AVPair anchor rules in some casesResolvedMarcos M

Actions
Related to Bug #14637: PHP shell script ``pfanchordrill`` shows duplicate anchor contentResolvedMarcos M

Actions
Actions #1

Updated by Jim Pingle 10 months ago

  • Project changed from pfSense Plus to pfSense
  • Subject changed from OpenVPN not removing cisco-avpair files in /tmp to OpenVPN not removing old Cisco-AVPair anchor rules and files in ``/tmp``
  • Category changed from OpenVPN to OpenVPN
  • Assignee set to Marcos M
  • Target version set to 2.8.0
  • Affected Plus Version deleted (23.05.1)
  • Plus Target Version set to 23.09
Actions #2

Updated by Jim Pingle 10 months ago

  • Related to Bug #12332: OpenVPN does not clear old Cisco-AVPair anchor rules in some cases added
Actions #3

Updated by Marcos M 10 months ago

I presume the "duplicate" ovpn_ovpns1_<user>_<port>.rules files differ by port number, in which case it'd mean the client has attempted the connection multiple times which could be due to an unstable connection. Or potentially simpler, simultaneous connections are being allowed.

The file "{$g['tmp_path']}/{$username}" is meant to be removed after the user has connected - I'm not sure yet why it would remain.

Actions #4

Updated by Michael Mercier 10 months ago

Yes, the "duplicate" ovpn_ovpns1_<user>_<port>.rules differ by port number, multiple connections are not enabled on the device.

Actions #5

Updated by Marcos M 10 months ago

  • Status changed from New to Feedback

The duplicate connections are disconnected automatically after the timeout period, at which point the related files/rules are removed. As for the user files persisting, try to recreate the issue after clearing out all of the temp files.

See below for a patch that will output some debugging info to /tmp/_test.txt. If you can recreate the issue, attach the debug info as well:

Show

Actions #6

Updated by Michael Mercier 10 months ago

Does OpenVPN need to be restarted after applying the patch? If so I will need to book a maintenance window for it to be applied.

Actions #7

Updated by Michael Mercier 9 months ago

I have been able to reproduce the issue, some details below.

My OpenVPN server has the Allow connected clients to retain their connections if their IP address changes. option enabled.

To reproduce the issue (with the above enabled):

1. Connect the device running the OpenVPN client to 'Network A', with public IP address 'pubip-A'
2. Connect to the OpenVPN server
3. Move the device to 'Network B', with public IP address 'pubip-B', making sure the OpenVPN connection doesn't timeout.
4. Disconnect from OpenVPN

This at a minimum:
1. Does not remove the /tmp/ovpn_<interface>_<user>_<port>.rules file
2. Leaves the anchor in the pf rules.

My exact steps:

1. Connect my laptop to SSID 'MYWORK'
2. Connect to OpenVPN server
3. Connect my laptop to SSID 'MYPHONE' - Personal hotspot on my phone
- verify I am still connected to OpenVPN by attempting to connect to a service
4. Disconnect from OpenVPN
5. On the pfSense head end

[23.05.1-RELEASE][root@vpn]/tmp:ls *user100*
ovpn_ovpns1_user100_65010.rules
[23.05.1-RELEASE][root@vpn]/tmp: pfctl -vsA | grep user100
  openvpn/ovpns1_user100_65010

As seen above, this didn't leave the /tmp/user100 file behind. This I have not yet figured out.

Actions #8

Updated by Michael Mercier 9 months ago

Logs from when I do the steps above:

ovpn_auth_verify: Call openvpn.tls-verify.php; returned OK [tls]
ovpn_auth_verify: Call openvpn.tls-verify.php; returned OK [tls]
ovpn_auth_verify: Call openvpn.tls-verify.php; returned OK [tls]
openvpn.attributes.php: Create /tmp/ovpn_ovpns1_user100_59225.rules
openvpn.auth-user.php: Call openvpn.attributes.php
openvpn.auth-user.php: Create /tmp/user100
ovpn_auth_verify_async: Call openvpn.auth-user.php; returned OK
openvpn.attributes.sh: Create /tmp/openvpn_cc_1e351be7ecdd5d9d663d8fda305672b8.tmp [client-connect]
openvpn.attributes.sh: Remove /tmp/user100 [client-connect]
openvpn.attributes.sh: Call openvpn.connect_async.sh
openvpn.connect_async.sh: Create /tmp/ovpn_ovpns1_user100_59225.lock [client-connect]
openvpn.connect_async.sh: Remove /tmp/ovpn_ovpns1_user100_59225.lock [client-connect]
openvpn.attributes.sh: Call openvpn.connect_async.sh

***Change to new Wi-fi access point (with different public IP), verify *internal* services are still available, then disconnect.***

openvpn.connect_async.sh: Create /tmp/ovpn_ovpns1_user100_6642.lock [client-disconnect]
openvpn.connect_async.sh: Remove /tmp/ovpn_ovpns1_user100_6642.lock [client-disconnect]
openvpn.connect_async.sh: Remove /tmp/ovpn_ovpns1_user100_6642.rules [client-disconnect]
openvpn.connect_async.sh: Remove /tmp/user100 [client-disconnect]

[23.05.1-RELEASE][root@vpn]/root: ls /tmp/*user100*
/tmp/ovpn_ovpns1_user100_59225.rules
[23.05.1-RELEASE][root@vpn.fieldeffect.com]/root: pfctl -a openvpn/ovpns1_user100_59225 -s rules
<rules assigned to user>
Actions #9

Updated by Michael Mercier 9 months ago

As for the route files (e.g. /tmp/user100) I see the following:

1. When some users login, the file is removed during connect_async. e.g.

 openvpn.auth-user.php: Create /tmp/user112
 openvpn.attributes.sh: Remove /tmp/user112 [client-connect]

2. With others, it is random

openvpn.auth-user.php: Create /tmp/user110
openvpn.connect_async.sh: Remove /tmp/user110 [client-disconnect]
openvpn.auth-user.php: Create /tmp/user110
openvpn.attributes.sh: Remove /tmp/user110 [client-connect]
openvpn.connect_async.sh: Remove /tmp/user110 [client-disconnect]

Actions #10

Updated by Marcos M 9 months ago

I've replicated the issue with the rules/anchors which I'll be looking at. The route file itself (/tmp/<User>) is always removed however - it may be that in your original case, the files already existed somehow.

Actions #11

Updated by Michael Mercier 9 months ago

Going back to the /tmp/<user> files.

I manually removed all the route (/tmp/<user>) files from the /tmp directory last week, some have reappeared.

e.g.

[23.05.1-RELEASE][root@vpn]/tmp: ls -al *user102*
-rw-r--r--  1 root  wheel  2338 Jul 24 14:34 ovpn_ovpns1_user102_1194.rules
-rw-r--r--  1 root  wheel   141 Jul 24 14:34 user102

Only some users have the files, and it only seems they exist is the user is connected (i.e. they seem to be deleted at logoff).

The output from the _test.txt logfile.

[23.05.1-RELEASE][root@vpn]/tmp: grep user102 _test.txt
openvpn.attributes.php: Create /tmp/ovpn_ovpns1_user102_1194.rules
openvpn.auth-user.php: Create /tmp/user102
openvpn.attributes.sh: Remove /tmp/user102 [client-connect]
openvpn.connect_async.sh: Create /tmp/ovpn_ovpns1_user102_1194.lock [client-connect]
openvpn.connect_async.sh: Remove /tmp/ovpn_ovpns1_user102_1194.lock [client-connect]
openvpn.attributes.php: Create /tmp/ovpn_ovpns1_user102_1194.rules
openvpn.auth-user.php: Create /tmp/user102
openvpn.attributes.php: Create /tmp/ovpn_ovpns1_user102_1194.rules
openvpn.auth-user.php: Create /tmp/user102
openvpn.attributes.php: Create /tmp/ovpn_ovpns1_user102_1194.rules
openvpn.auth-user.php: Create /tmp/user102
openvpn.attributes.php: Create /tmp/ovpn_ovpns1_user102_1194.rules
openvpn.auth-user.php: Create /tmp/user102
openvpn.attributes.php: Create /tmp/ovpn_ovpns1_user102_1194.rules
openvpn.auth-user.php: Create /tmp/user102
openvpn.attributes.php: Create /tmp/ovpn_ovpns1_user102_1194.rules
openvpn.auth-user.php: Create /tmp/user102
Actions #12

Updated by Marcos M 9 months ago

  • Status changed from Feedback to Needs Patch
  • Target version changed from 2.8.0 to CE-Next
  • Plus Target Version changed from 23.09 to Plus-Next

The duplicate rules listed with pfanchordrill are a cosmetic issue - see #14637.

As for the files that aren't being cleaned up, the OpenVPN daemon doesn't have a script hook for floating clients; this issue will need to wait until an upstream bug/feature is resolved.

Actions #13

Updated by Marcos M 9 months ago

  • Related to Bug #14637: PHP shell script ``pfanchordrill`` shows duplicate anchor content added
Actions #14

Updated by Michael Mercier 9 months ago

I have to disagree that they are a cosmetic issue.

This issue was originally discovered via the following:
1. A new service was deployed in my organization - IP 10.10.10.10
2. No Cisco-AVPair was added to the VPN for it to be accessible to the targeted users
3. One of the targeted users (user102) was able to access the new service over the VPN
4. Other targeted users attempted to access the new service over the VPN without success

What looks like happened on the head end.
1. A user (User110) with more privileges had logged into the VPN (with a Cisco-AVPair for the subnet (10.10.10.0/24) containing the IP address of the new service)
2. User110 was disconnected in a unknown, non-standard way, which left an anchor installed in the pf rules
3. User102 connected to the VPN, being assigned the same IP address as User110 has been previously assigned
4. User102 was now hitting the anchor left behind by User110, giving them access to 10.10.10.10. This would not have been possible with the Cisco-AVPair rules pushed by RADIUS

I'm keeping an eye on pfSense to see if I can provide a concrete example. I need to do some maintenance to the OpenVPN setting in the next couple of days, I will reboot the device at the same time to start in a fresh state.

Please let me know your thoughts.

Actions #15

Updated by Marcos M 9 months ago

Until the referenced functionality is added upstream, floating client support will need to be disabled if avpair rules need to be used - otherwise there's the potential for the issues you're seeing. If you are able to replicate an issue with floating support disabled, please provide steps to replicate it.

Actions #16

Updated by Michael Mercier 7 months ago

Marcos M wrote in #note-15:

Until the referenced functionality is added upstream, floating client support will need to be disabled if avpair rules need to be used - otherwise there's the potential for the issues you're seeing. If you are able to replicate an issue with floating support disabled, please provide steps to replicate it.

Hi Marcos,

I now have the OpenVPN server running with the 'Client Settings -> Dynamic IP' option disabled. I am still seeing multiple files for users in the /tmp directory.

e.g.
ovpn_ovpns1_user102_49732.rules
ovpn_ovpns1_user102_50633.rules
ovpn_ovpns1_user102_57634.rules

I also see the 'user102' file that contains the 'push "route..." options.

I haven't had a chance to see if I can reproduce this issue yet, just want to update.

Thanks,
Mike

Actions

Also available in: Atom PDF