pfSense

I found the following bug in pfSense 2.0-BETA4 (i386) built on Tue Dec 21 15:02:48 EST 2010.

I setup an OpenVPN server with Server Mode set to "Peer to Peer (SSL/TLS)" and with TLS Authentication enabled (i.e. "Enable authentication of TLS packets" is checked).

The tunnel did not come up. Checking the OpenVPN logs showed this message on both client and server:

Jan 14 12:15:36     openvpn[52724]: TLS Error: incoming packet authentication failed from [AF_INET]216.123.209.88:41869
Jan 14 12:15:41     openvpn[52724]: Authenticate/Decrypt packet error: packet HMAC authentication failed

I triple-checked the TLS authentication key on both sides and they matched. Googling the error suggested that both sides were probably wrongly using the same number (either both 0 or both 1) at the end of the tls-auth line in the OpenVPN config file. Sure enough, when I looked at the conf file in /var/etc/openvpn on both the client and the server, both of them had 1 at the end of the tls-auth line.

For example, on the server, the line was:

tls-auth /var/etc/openvpn/server2.tls-auth 1

However, according to the OpenVPN How-to, the convention is for the server to use 0 and the client to use 1:

In the server configuration, add:

    tls-auth ta.key 0

In the client configuration, add:

    tls-auth ta.key 1

When I manually edited the server config file, changing 1 to 0, and then restarted the server, the TLS error went away and the tunnel came up. However, upon rebooting, the setting got set back to 1 because I guess the config file gets rebuilt.

Interestingly, if I changed the Server Mode to "Remote Access (SSL/TLS)", the setting for tls-auth was correctly set to 0 on the server side.

So I think this is a bug. To fix it, the server config file that gets generated by pfSense should use 0 at the end of the tls-auth line for Server Mode "Peer to Peer (SSL/TLS)".