Project

General

Profile

Bug #754

hifn driver and AES192 and 256

Added by Mr Horizontal about 9 years ago. Updated over 4 years ago.

Status:
Needs Patch
Priority:
Normal
Assignee:
-
Category:
Operating System
Target version:
-
Start date:
07/20/2010
Due date:
% Done:

0%

Estimated time:
Affected Version:
2.0
Affected Architecture:

Description

Even though it's an old driver from FreeBSD 6.0 and even though I've tried submitting to the lovely FreeBSD people about this, it seems there are plenty of patches to ensure that the hifn driver for Soekris vpn1411 cards support aes-192-cbc and aes-256-cbc, but they've not been put into the mainstream FreeBSD build tree.

As AES-256 is such a popular algo and currently an OpenVPN tunnel using AES256 will only work at 4.5mbps in pfSense 2.0 on an ALIX (using the CPU), can you please investigate whether it's possible to find a patch and make it work for AES-192 and AES-256 in pfSense, or at the very least ask the FreeBSD people under the auspices of the pfSense project instead of a lowly pfSense user like myself to fix it (as my request from January was obviously ignored).

This will make life using ALIX boxes with vpn1411 cards in them a very viable VPN platform!

test1.txt (5.36 KB) test1.txt Test 1 (Placebo) Mr Horizontal, 03/05/2011 07:49 AM
test2 (6.24 KB) test2 Test 2 (GLXSB) Mr Horizontal, 03/05/2011 07:49 AM
test3 (11.7 KB) test3 Test 3 (Hifn) Mr Horizontal, 03/05/2011 07:49 AM
Soekris5501_VPN1401(HIFN 7955)-2.03R.txt (13.1 KB) Soekris5501_VPN1401(HIFN 7955)-2.03R.txt Scott Douglas, 09/28/2013 08:41 AM
Soekris5501_VPN1401(HIFN 7955)-2.1R (13.2 KB) Soekris5501_VPN1401(HIFN 7955)-2.1R Scott Douglas, 09/28/2013 08:41 AM

History

#1 Updated by Chris Buechler about 9 years ago

  • Category changed from OpenVPN to Operating System

where is this patch?

#2 Updated by Chris Buechler about 9 years ago

  • Status changed from New to Feedback

Needs testing on 8.1 and a pointer to the mentioned patch if such a thing exists.

#3 Updated by Mr Horizontal about 9 years ago

I'm no expert at knowing whether this is any good - you have to trawl through the archives, but this one seems pretty promising:

http://gnats.netbsd.org/26673

#4 Updated by Mr Horizontal about 9 years ago

8.1 hasn't changed anything. Here's the output on an ALIX 2D3:

  1. uname -a
    FreeBSD gateway.X.X.X 8.1-RC2 FreeBSD 8.1-RC2 #0: Mon Jul 26 06:24:21 EDT 2010 sullrich@FreeBSD_8.0_pfSense_2.0-snaps.pfsense.org:/usr/obj.pfSense/usr/pfSensesrc/src/sys/pfSense_wrap.8.i386 i386 #
  2. openssl engine -c
    (cryptodev) BSD cryptodev engine
    [RSA, DSA, DH, DES-CBC, DES-EDE3-CBC, AES-128-CBC]
    (padlock) VIA PadLock (no-RNG, no-ACE)
    (dynamic) Dynamic engine loading support #
  3. openssl speed -evp aes-256-cbc -engine cryptodev
    engine "cryptodev" set.
    To get the most accurate results, try to run this
    program when this computer is idle.
    Doing aes-256-cbc for 3s on 16 size blocks: 736145 aes-256-cbc's in 2.99s
    Doing aes-256-cbc for 3s on 64 size blocks: 202072 aes-256-cbc's in 2.97s
    Doing aes-256-cbc for 3s on 256 size blocks: 52063 aes-256-cbc's in 2.98s
    Doing aes-256-cbc for 3s on 1024 size blocks: 13132 aes-256-cbc's in 2.98s
    Doing aes-256-cbc for 3s on 8192 size blocks: 1643 aes-256-cbc's in 2.98s
    OpenSSL 0.9.8n 24 Mar 2010
    built on: date not available
    options:bn(64,32) md2(int) rc4(idx,int) des(ptr,risc1,16,long) aes(partial) blowfish(idx)
    compiler: cc
    available timing options: USE_TOD HZ=128 [sysconf value]
    timing function used: getrusage
    The 'numbers' are in 1000s of bytes per second processed.
    type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes
    aes-256-cbc 3939.52k 4357.76k 4470.99k 4514.63k 4519.40k

#5 Updated by Mr Horizontal about 9 years ago

There's also a FreeBSD bug report relating to this that's been open since Feb 2008: http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/120270

#6 Updated by Mr Horizontal almost 9 years ago

This is still not fixed as of 10 Nov 2010...

#7 Updated by Chris Buechler almost 9 years ago

  • Status changed from Feedback to New

need to determine if there is a reason this patch in kern/120270 hasn't been committed.

#8 Updated by Scott Ullrich almost 9 years ago

  • Target version changed from 2.0 to 2.1

Still not fixed in FreeBSD. Delaying ticket to 2.1.

#9 Updated by Chris Buechler almost 9 years ago

  • Target version changed from 2.1 to 2.0

We need to follow up on the PR above, a customer needs this.

#10 Updated by Ermal Lu├ži over 8 years ago

  • Status changed from New to Feedback

This has been committed to repo.
Please test.

#11 Updated by Dominik Schips over 8 years ago

I did a test with my alix 2d2 box and a vpn1411 card.

Here are the results with the fresh updated image (pfSense-2.0-RC1-4g-i386-20110214-0324-nanobsd-upgrade.img.gz) on the alix box:

root(1): openssl engine -c
(cryptodev) BSD cryptodev engine
[RSA, DSA, DH, DES-CBC, DES-EDE3-CBC, AES-128-CBC]
(padlock) VIA PadLock (no-RNG, no-ACE)
(dynamic) Dynamic engine loading support

First test with aes-128-cbc:

/root(4): openssl speed -evp aes-128-cbc -engine cryptodev
engine "cryptodev" set.
To get the most accurate results, try to run this
program when this computer is idle.
Doing aes-128-cbc for 3s on 16 size blocks: 59274 aes-128-cbc's in 0.08s
Doing aes-128-cbc for 3s on 64 size blocks: 50889 aes-128-cbc's in 0.07s
Doing aes-128-cbc for 3s on 256 size blocks: 46414 aes-128-cbc's in 0.04s
Doing aes-128-cbc for 3s on 1024 size blocks: 28853 aes-128-cbc's in 0.04s
Doing aes-128-cbc for 3s on 8192 size blocks: 6377 aes-128-cbc's in 0.01s
OpenSSL 0.9.8n 24 Mar 2010
built on: date not available
options:bn(64,32) md2(int) rc4(idx,int) des(ptr,risc1,16,long) aes(partial) blowfish(idx)
compiler: cc
available timing options: USE_TOD HZ=128 [sysconf value]
timing function used: getrusage
The 'numbers' are in 1000s of bytes per second processed.
type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes
aes-128-cbc 11926.96k 43881.06k 292746.23k 799087.79k 4099857.48k

Second test with aes-192-cbc:

/root(5): openssl speed -evp aes-192-cbc -engine cryptodev
engine "cryptodev" set.
To get the most accurate results, try to run this
program when this computer is idle.
Doing aes-192-cbc for 3s on 16 size blocks: 796761 aes-192-cbc's in 2.90s
Doing aes-192-cbc for 3s on 64 size blocks: 223249 aes-192-cbc's in 2.93s
Doing aes-192-cbc for 3s on 256 size blocks: 58179 aes-192-cbc's in 2.95s
Doing aes-192-cbc for 3s on 1024 size blocks: 14786 aes-192-cbc's in 2.98s
Doing aes-192-cbc for 3s on 8192 size blocks: 1835 aes-192-cbc's in 2.95s
OpenSSL 0.9.8n 24 Mar 2010
built on: date not available
options:bn(64,32) md2(int) rc4(idx,int) des(ptr,risc1,16,long) aes(partial) blowfish(idx)
compiler: cc
available timing options: USE_TOD HZ=128 [sysconf value]
timing function used: getrusage
The 'numbers' are in 1000s of bytes per second processed.
type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes
aes-192-cbc 4400.92k 4871.91k 5042.90k 5083.21k 5104.08k

Third test with aes-256-cbc:

/root(2): openssl speed -evp aes-256-cbc -engine cryptodev
engine "cryptodev" set.
To get the most accurate results, try to run this
program when this computer is idle.
Doing aes-256-cbc for 3s on 16 size blocks: 726153 aes-256-cbc's in 2.94s
Doing aes-256-cbc for 3s on 64 size blocks: 190858 aes-256-cbc's in 2.81s
Doing aes-256-cbc for 3s on 256 size blocks: 49574 aes-256-cbc's in 2.84s
Doing aes-256-cbc for 3s on 1024 size blocks: 12053 aes-256-cbc's in 2.75s
Doing aes-256-cbc for 3s on 8192 size blocks: 1473 aes-256-cbc's in 2.69s
OpenSSL 0.9.8n 24 Mar 2010
built on: date not available
options:bn(64,32) md2(int) rc4(idx,int) des(ptr,risc1,16,long) aes(partial) blowfish(idx)
compiler: cc
available timing options: USE_TOD HZ=128 [sysconf value]
timing function used: getrusage
The 'numbers' are in 1000s of bytes per second processed.
type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes
aes-256-cbc 3953.05k 4343.71k 4476.49k 4492.25k 4482.33k

Looks like it doesn't work for me. Or did I something wrong at the tests?

#12 Updated by F S over 8 years ago

Test System: alix 2d2 + vpn1411 card.
Beta Version: pfSense-2.0-RC1-4g-i386-20110214-0324

seems like the vpn card is used according to cryptostats. But the shuffeling from kernel to userland and back again takes all the performance gains. With OpenVPN set to aes-128-cbc and using cryptodev, the max speed was approx. 10-12Mbit which is with a CPU usage of 90% same as it would be on bare metal. Maybe I missing some other configuration to make this work propper?!?

  1. cryptostats
    30253149 symmetric crypto ops (0 errors, 0 times driver blocked)
    0 key ops (0 errors, 0 times driver blocked)
    14967680 crypto dispatch thread activations
    0 crypto return thread activations

#13 Updated by Jim Pingle over 8 years ago

F S wrote:

Test System: alix 2d2 + vpn1411 card.
Beta Version: pfSense-2.0-RC1-4g-i386-20110214-0324

seems like the vpn card is used according to cryptostats. But the shuffeling from kernel to userland and back again takes all the performance gains. With OpenVPN set to aes-128-cbc and using cryptodev, the max speed was approx. 10-12Mbit which is with a CPU usage of 90% same as it would be on bare metal. Maybe I missing some other configuration to make this work propper?!?

Did you disable glxsb under System > Advanced, on the misc tab? Unless you disable glxsb, the Hifn won't be used.

#14 Updated by Dominik Schips over 8 years ago

I did the test with and without glxsb activated. There was no difference in the results.

Could it be that the hifn driver isn't loaded correct?
How could I check this?

#15 Updated by Alexander Norberg over 8 years ago

pfSense 2.0-RC1-nanoBSD (i386) built on Sat Feb 26 16:33:51 EST 2011
Running on Soekris net5501 with Hardware crypto card Hifn 7955

[2.0-RC1][]/root(1): openssl speed -evp aes-128-cbc -engine cryptodev
engine "cryptodev" set.
To get the most accurate results, try to run this
program when this computer is idle.
Doing aes-128-cbc for 3s on 16 size blocks: 18183 aes-128-cbc's in 0.06s
Doing aes-128-cbc for 3s on 64 size blocks: 17926 aes-128-cbc's in 0.07s
Doing aes-128-cbc for 3s on 256 size blocks: 16731 aes-128-cbc's in 0.04s
Doing aes-128-cbc for 3s on 1024 size blocks: 13808 aes-128-cbc's in 0.06s
Doing aes-128-cbc for 3s on 8192 size blocks: 5112 aes-128-cbc's in 0.01s
OpenSSL 0.9.8n 24 Mar 2010
built on: date not available
options:bn(64,32) md2(int) rc4(idx,int) des(ptr,risc1,16,long) aes(partial) blowfish(idx)
compiler: cc
available timing options: USE_TOD HZ=128 [sysconf value]
timing function used: getrusage
The 'numbers' are in 1000s of bytes per second processed.
type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes
aes-128-cbc 5133.81k 16332.56k 106439.76k 224199.92k 5177732.94k

[2.0-RC1][]/root(2): openssl speed -evp aes-192-cbc -engine cryptodev
engine "cryptodev" set.
To get the most accurate results, try to run this
program when this computer is idle.
Doing aes-192-cbc for 3s on 16 size blocks: 788966 aes-192-cbc's in 2.85s
Doing aes-192-cbc for 3s on 64 size blocks: 219687 aes-192-cbc's in 2.85s
Doing aes-192-cbc for 3s on 256 size blocks: 55989 aes-192-cbc's in 2.83s
Doing aes-192-cbc for 3s on 1024 size blocks: 13800 aes-192-cbc's in 2.78s
Doing aes-192-cbc for 3s on 8192 size blocks: 1666 aes-192-cbc's in 2.69s
OpenSSL 0.9.8n 24 Mar 2010
built on: date not available
options:bn(64,32) md2(int) rc4(idx,int) des(ptr,risc1,16,long) aes(partial) blowfish(idx)
compiler: cc
available timing options: USE_TOD HZ=128 [sysconf value]
timing function used: getrusage
The 'numbers' are in 1000s of bytes per second processed.
type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes
aes-192-cbc 4423.20k 4925.50k 5068.97k 5080.86k 5082.34k

[2.0-RC1][]/root(3): openssl speed -evp aes-256-cbc -engine cryptodev
engine "cryptodev" set.
To get the most accurate results, try to run this
program when this computer is idle.
Doing aes-256-cbc for 3s on 16 size blocks: 731079 aes-256-cbc's in 2.97s
Doing aes-256-cbc for 3s on 64 size blocks: 202640 aes-256-cbc's in 2.98s
Doing aes-256-cbc for 3s on 256 size blocks: 52026 aes-256-cbc's in 2.96s
Doing aes-256-cbc for 3s on 1024 size blocks: 12135 aes-256-cbc's in 2.75s
Doing aes-256-cbc for 3s on 8192 size blocks: 1643 aes-256-cbc's in 2.98s
OpenSSL 0.9.8n 24 Mar 2010
built on: date not available
options:bn(64,32) md2(int) rc4(idx,int) des(ptr,risc1,16,long) aes(partial) blowfish(idx)
compiler: cc
available timing options: USE_TOD HZ=128 [sysconf value]
timing function used: getrusage
The 'numbers' are in 1000s of bytes per second processed.
type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes
aes-256-cbc 3942.66k 4356.92k 4492.45k 4511.50k 4518.60k

[2.0-RC1][]/root(4): cryptostats
71914 symmetric crypto ops (0 errors, 0 times driver blocked)
0 key ops (0 errors, 0 times driver blocked)
0 crypto dispatch thread activations
154 crypto return thread activations

[2.0-RC1][]/root(5): openssl engine -c
(cryptodev) BSD cryptodev engine
[RSA, DSA, DH, DES-CBC, DES-EDE3-CBC, AES-128-CBC]
(padlock) VIA PadLock (no-RNG, no-ACE)
(dynamic) Dynamic engine loading support

If more tests are needed, please let me know.

#16 Updated by Mr Horizontal over 8 years ago

Below I've posted results for the following series of commands, so you get a summary as well as all the output:

  1. openssl engine -c
  2. cryptostats
  3. hifnstats
  4. cryptotest -z
  5. cryptostats
  6. hifnstats
  7. openssl speed -evp aes-128-cbc -engine cryptodev
  8. cryptostats
  9. hifnstats
  10. openssl speed -evp aes-192-cbc -engine cryptodev
  11. cryptostats
  12. hifnstats
  13. openssl speed -evp aes-256-cbc -engine cryptodev
  14. cryptostats
  15. hifnstats

I then conducted 3 tests each after a clean reboot on an Alix 2D2. The hifn card is a Soekris vpn1411 card:
[Test 1]: disable glxsb and remove hifn.
[Test 2]: enable glxsb and remove hifn.
[Test 3]: disable glxsb and insert hifn.

Results are attached if you want to read through the output.

Test 1 and 2 work as expected, with the CPU being slower than the glxsb acceleration. This is backed up by the fact that cryptostats shows activity, while hifnstats shows nothing happening.

Test 3 shows that AES 128 is accelerated, but 192 and 256 don't do anything.

In the meantime, cryptotest -z tests all the functions of the HIFN card withoud using OpenSSL, so...

- There are 3 parts where the hifn wouldn't be working:
- The hifn(4) driver itself, but I don't expect this to be bad since cryptotest z works
OpenSSL 0.9.8n seems to have the new engine table format, see
http://cvs.openssl.org/rlog?f=openssl/crypto/engine/eng_cryptodev.c
which may break support for hifn, given that it has recently been touched for adding direct
AES-NI support in Intel Westmere and later CPUs
- To use hifn, OpenSSL loads the BSD cryptodev engine which may also have broken support, as the
latest cryptodev engine code in FreeBSD's source tree is quite new:
FreeBSD source shows:
http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/opencrypto/cryptodev.h
http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/opencrypto/cryptodev.c
compared to another source, in DragonflyBSD:
http://cvsweb.dragonflybsd.org/cvsweb/src/sys/opencrypto/cryptodev.h
http://cvsweb.dragonflybsd.org/cvsweb/src/sys/opencrypto/cryptodev.c

Alternatively, given all the above, they basically require compiler flags to be set correctly and could have been compiled wrong?

#17 Updated by Chris Buechler about 8 years ago

  • Target version deleted (2.0)

#18 Updated by Stephen Gaglairdo about 7 years ago

Just would like to confirm that fairly identical results are produced on my ALIX.6F2 with my soekris VPN1411 (HIFN 7955). Reproduces exactly as these posters have said, with nearly identical numbers when I access via serial port and run them.

Has there been any progress on this for the next stable release (whenever that should be)?

#19 Updated by Igor Liebermann about 6 years ago

I can confirm the problem on pfSense 2.0.3, Alix.2D13 and soekris VPN1411 (hifn 7955).
There is no difference in throughput in the OpenVPN AES256-Tunnel.

Is it planned to fix this issue? Any target release?

#20 Updated by Scott Douglas almost 6 years ago

I've attached a couple of text files showing the Soekris VPN-1401 (Hifn-7955 / PCI Bus) version working in a Soekris 5501-70.

- Soekris VPN-1401 (PCI Bus) and a Soekris 5501-70 seems to be working OK.
- Soekris VPN-1411 (mini-PCI bus) and an ALIX.2D2 or ALIX.6F2 does not seem to be working.

What is the output from the command, "dmesg | grep -i hifn" on the various ALIX systems?

My Soekris 5501-70 shows the following output for both v2.03-RELEASE and v2.1-RELEASE:

root(17): dmesg | grep -i hifn
hifn0 mem 0xa0005000-0xa0005fff,0xa0006000-0xa0007fff,0xa0008000-0xa000ffff irq 10 at device 14.0 on pci0
hifn0: [ITHREAD]
hifn0: Hifn 7955, rev 0, 32KB dram, pll=0x801<ext clk, 4x mult>

Is there anything else that you need tested?

#21 Updated by Chris Buechler over 4 years ago

  • Status changed from Feedback to Needs Patch

if someone wants to put the efforts into fixing this (if it isn't already on 2.2x with FreeBSD 10.1 base), please pursue and get fixed upstream in FreeBSD.

Also available in: Atom PDF