Project

General

Profile

Actions
Copy link

Regression #13613

closed

OpenVPN crashes due to if_tuntap changes

Added by Marcos M about 2 years ago. Updated almost 2 years ago.

Status:
Resolved
Priority:
High
Assignee:
Kristof Provost
Category:
OpenVPN
Target version:
23.01
Start date:
Due date:
% Done:

0%

Estimated time:
Release Notes:
Force Exclusion
Affected Plus Version:
Affected Architecture:

Description

Tested on pfSense-23.01.a.20221031.0600.

Client/Server (no DCO) crashes only after a reboot - starting it manually afterwards works:

Oct 31 10:36:55     openvpn     18667     OpenVPN 2.6_git amd64-portbld-freebsd14.0 [SSL (OpenSSL)] [LZO] [LZ4] [MH/RECVDA] [AEAD] [DCO]
Oct 31 10:36:55     openvpn     18667     library versions: OpenSSL 1.1.1q-freebsd 5 Jul 2022, LZO 2.10
Oct 31 10:36:55     openvpn     18667     NOTE: the current --script-security setting may allow this configuration to call user-defined scripts
Oct 31 10:36:55     openvpn     18667     Initializing OpenSSL support for engine 'rdrand'
Oct 31 10:36:55     openvpn     18667     WARNING: experimental option --capath /var/etc/openvpn/server1/ca
Oct 31 10:36:55     openvpn     18667     Using random OpenVPN auth-token server key.
Oct 31 10:36:55     openvpn     18667     TUN/TAP device ovpns1 exists previously, keep at program end
Oct 31 10:36:55     openvpn     18667     TUN/TAP device /dev/tun1 opened
Oct 31 10:36:55     openvpn     18667     ioctl(TUNSIFMODE): Device busy (errno=16)
Oct 31 10:36:55     openvpn     18667     /sbin/ifconfig ovpns1 172.25.1.1/24 mtu 1500 up
Oct 31 10:36:55     openvpn     18667     FreeBSD ifconfig failed: external program exited with error status: 1
Oct 31 10:36:55     openvpn     18667     Exiting due to fatal error
Oct 31 10:39:07     openvpn     74815     OpenVPN 2.6_git amd64-portbld-freebsd14.0 [SSL (OpenSSL)] [LZO] [LZ4] [MH/RECVDA] [AEAD] [DCO]
Oct 31 10:39:07     openvpn     74815     library versions: OpenSSL 1.1.1q-freebsd 5 Jul 2022, LZO 2.10
Oct 31 10:39:07     openvpn     74815     NOTE: the current --script-security setting may allow this configuration to call user-defined scripts
Oct 31 10:39:07     openvpn     74815     Initializing OpenSSL support for engine 'rdrand'
Oct 31 10:39:07     openvpn     74815     WARNING: experimental option --capath /var/etc/openvpn/server1/ca
Oct 31 10:39:07     openvpn     74815     Using random OpenVPN auth-token server key.
Oct 31 10:39:07     openvpn     74815     TUN/TAP device ovpns1 exists previously, keep at program end
Oct 31 10:39:07     openvpn     74815     TUN/TAP device /dev/tun1 opened
Oct 31 10:39:07     openvpn     74815     /sbin/ifconfig ovpns1 172.25.1.1/24 mtu 1500 up
Oct 31 10:39:07     openvpn     74815     /usr/local/sbin/ovpn-linkup ovpns1 1500 0 172.25.1.1 255.255.255.0 init
Oct 31 10:39:07     openvpn     74815     UDPv4 link local (bound): [AF_INET]127.0.0.1:1195
Oct 31 10:39:07     openvpn     74815     UDPv4 link remote: [AF_UNSPEC]
Oct 31 10:39:07     openvpn     74815     Initialization Sequence Completed

Actions
Copy link
 #2

Updated by Marcos M about 2 years ago

  • Status changed from Ready To Test to Resolved

Tested patch - issue now fixed.

Actions
Copy link
 #3

Updated by Marcos M almost 2 years ago

  • Status changed from Resolved to Feedback

I just ran into a different way of triggering what seems to be a similar issue. Editing a client with DCO enabled, unchecking DCO, then Saving/Applying:

Nov 29 23:36:45     openvpn     10586     event_wait : Interrupted system call (fd=-1,code=4)
Nov 29 23:36:45     openvpn     10586     SIGTERM received, sending exit notification to peer
Nov 29 23:36:45     openvpn     10586     Attempting to send data packet while data channel offload is in use. Dropping packet
Nov 29 23:36:45     openvpn     10586     Failed to poll for packets: Device not configured (errno=6)
Nov 29 23:36:45     openvpn     10586     Failed to poll for packets: Device not configured (errno=6)
Nov 29 23:36:46     openvpn     17080     WARNING: file '/var/etc/openvpn/client3/up' is group or others accessible
Nov 29 23:36:46     openvpn     17080     OpenVPN 2.6_git amd64-portbld-freebsd14.0 [SSL (OpenSSL)] [LZO] [LZ4] [MH/RECVDA] [AEAD] [DCO]
Nov 29 23:36:46     openvpn     17080     library versions: OpenSSL 1.1.1q-freebsd 5 Jul 2022, LZO 2.10
Nov 29 23:36:46     openvpn     17080     MANAGEMENT: unix domain socket listening on /var/etc/openvpn/client3/sock
Nov 29 23:36:46     openvpn     17080     NOTE: the current --script-security setting may allow this configuration to call user-defined scripts
Nov 29 23:36:46     openvpn     17080     Initializing OpenSSL support for engine 'rdrand'
Nov 29 23:36:46     openvpn     17080     WARNING: experimental option --capath /var/etc/openvpn/client3/ca
Nov 29 23:36:46     openvpn     17080     Outgoing Control Channel Authentication: Using 256 bit message hash 'SHA256' for HMAC authentication
Nov 29 23:36:46     openvpn     17080     Incoming Control Channel Authentication: Using 256 bit message hash 'SHA256' for HMAC authentication
Nov 29 23:36:46     openvpn     17080     TCP/UDP: Preserving recently used remote address: [AF_INET]<SERVER_IP>:1194
Nov 29 23:36:46     openvpn     17080     Socket Buffers: R=[42080->42080] S=[57344->57344]
Nov 29 23:36:46     openvpn     17080     UDPv4 link local (bound): [AF_INET]127.0.0.1:0
Nov 29 23:36:46     openvpn     17080     UDPv4 link remote: [AF_INET]<SERVER_IP>:1194
Nov 29 23:36:46     openvpn     17080     TLS: Initial packet from [AF_INET]<SERVER_IP>:1194, sid=ddda52d8 1821a463
Nov 29 23:36:46     openvpn     10586     Failed to delete peer: Invalid argument (errno=22)
Nov 29 23:36:46     openvpn     10586     /sbin/route delete -net 172.17.105.0 172.17.5.1 255.255.255.0
Nov 29 23:36:46     openvpn     10586     ERROR: FreeBSD route delete command failed: external program exited with error status: 1
Nov 29 23:36:46     openvpn     10586     Closing DCO interface
Nov 29 23:36:46     openvpn     10586     /sbin/ifconfig ovpnc3 172.17.5.2 -alias
Nov 29 23:36:46     openvpn     10586     FreeBSD ip addr del failed: external program exited with error status: 1
Nov 29 23:36:46     openvpn     10586     /sbin/ifconfig ovpnc3 destroy
Nov 29 23:36:47     openvpn     10586     /usr/local/sbin/ovpn-linkdown ovpnc3 1500 0 172.17.5.2 255.255.255.0 init
Nov 29 23:36:47     openvpn     21022     Flushing states on OpenVPN interface ovpnc3 (Link Down)
Nov 29 23:36:47     openvpn     17080     VERIFY WARNING: depth=0, unable to get certificate CRL: CN=gw.<SERVER_DOMAIN>
Nov 29 23:36:47     openvpn     17080     VERIFY WARNING: depth=1, unable to get certificate CRL: CN=<SERVER_DOMAIN>-internal
Nov 29 23:36:47     openvpn     17080     VERIFY OK: depth=1, CN=<SERVER_DOMAIN>-internal
Nov 29 23:36:47     openvpn     17080     VERIFY KU OK
Nov 29 23:36:47     openvpn     17080     Validating certificate extended key usage
Nov 29 23:36:47     openvpn     17080     ++ Certificate has EKU (str) TLS Web Server Authentication, expects TLS Web Server Authentication
Nov 29 23:36:47     openvpn     17080     VERIFY EKU OK
Nov 29 23:36:47     openvpn     17080     VERIFY OK: depth=0, CN=gw.<SERVER_DOMAIN>
Nov 29 23:36:47     openvpn     10586     SIGTERM[soft,exit-with-notification] received, process exiting
Nov 29 23:36:48     openvpn     17080     Control Channel: TLSv1.3, cipher TLSv1.3 TLS_AES_256_GCM_SHA384, peer certificate: 2048 bit RSA, signature: RSA-SHA256
Nov 29 23:36:48     openvpn     17080     [gw.<SERVER_DOMAIN>] Peer Connection Initiated with [AF_INET]<SERVER_IP>:1194
Nov 29 23:36:49     openvpn     17080     SENT CONTROL [gw.<SERVER_DOMAIN>]: 'PUSH_REQUEST' (status=1)
Nov 29 23:36:49     openvpn     17080     PUSH: Received control message: 'PUSH_REPLY,route 10.0.5.0 255.255.255.0,dhcp-option DOMAIN <SERVER_DOMAIN>,dhcp-option DNS 172.17.5.1,route 172.17.105.0 255.255.255.0,client-nat dnat 172.17.105.0 255.255.255.0 10.0.5.0,route-gateway 172.17.5.1,topology subnet,ping 10,ping-restart 60,ifconfig 172.17.5.2 255.255.255.0,peer-id 1,cipher AES-256-GCM,key-derivation tls-ekm'
Nov 29 23:36:49     openvpn     17080     Options error: option 'route' cannot be used in this context ([PUSH-OPTIONS])
Nov 29 23:36:49     openvpn     17080     Options error: option 'dhcp-option' cannot be used in this context ([PUSH-OPTIONS])
Nov 29 23:36:49     openvpn     17080     Options error: option 'dhcp-option' cannot be used in this context ([PUSH-OPTIONS])
Nov 29 23:36:49     openvpn     17080     Options error: option 'route' cannot be used in this context ([PUSH-OPTIONS])
Nov 29 23:36:49     openvpn     17080     Options error: option 'client-nat' cannot be used in this context ([PUSH-OPTIONS])
Nov 29 23:36:49     openvpn     17080     OPTIONS IMPORT: timers and/or timeouts modified
Nov 29 23:36:49     openvpn     17080     OPTIONS IMPORT: --ifconfig/up options modified
Nov 29 23:36:49     openvpn     17080     OPTIONS IMPORT: route-related options modified
Nov 29 23:36:49     openvpn     17080     OPTIONS IMPORT: peer-id set
Nov 29 23:36:49     openvpn     17080     OPTIONS IMPORT: data channel crypto options modified
Nov 29 23:36:49     openvpn     17080     ROUTE_GATEWAY 177.231.47.1/255.255.255.0 IFACE=vmx0.99 HWADDR=00:50:56:b2:85:60
Nov 29 23:36:49     openvpn     17080     TUN/TAP device /dev/tun3 opened
Nov 29 23:36:49     openvpn     17080     /sbin/ifconfig ovpnc3 172.17.5.2/24 mtu 1500 up
Nov 29 23:36:49     openvpn     17080     FreeBSD ifconfig failed: external program exited with error status: 1
Nov 29 23:36:49     openvpn     17080     Exiting due to fatal error
Actions
Copy link
 #4

Updated by Kristof Provost almost 2 years ago

I can reproduce that here. It looks like the problem is that we send a SIGTERM to openvpn, but don't wait until it actually exits before destroying the interface. That it turn causes it to not actually exit, breaking the subsequent openvpn instance.

https://gitlab.netgate.com/pfSense/factory/-/merge_requests/87 should fix that.

Actions
Copy link
 #5

Updated by Marcos M almost 2 years ago

Tested patch and it worked well here.

Actions
Copy link
 #6

Updated by Jim Pingle almost 2 years ago

  • Status changed from Feedback to Pull Request Review
Actions
Copy link
 #7

Updated by Kristof Provost almost 2 years ago

  • Status changed from Pull Request Review to Ready To Test

Merged. This will turn up in the next snapshot build.

Actions
Copy link
 #8

Updated by Jim Pingle almost 2 years ago

  • Project changed from pfSense to pfSense Plus
  • Category changed from OpenVPN to OpenVPN
  • Status changed from Ready To Test to Feedback
  • Target version set to 23.01
  • Plus Target Version deleted (23.01)
Actions
Copy link
 #9

Updated by Jim Pingle almost 2 years ago

  • Subject changed from OpenVPN crashes due to if_tuntap changes. to OpenVPN crashes due to if_tuntap changes
Actions
Copy link
 #10

Updated by Marcos M almost 2 years ago

  • Status changed from Feedback to Resolved
Actions
Copy link

Also available in: Atom PDF