pfSense

Repeated the same tests with different combination of HW:

SG4860 <--> SG2440 with enabled Async crypto on both sides - no problem with SMB traffic flow. Test OK.

SG4860 <--> SG3100 with enabled Async crypto on both sides - SMB is not being passed through. Disabled it only on 3100 - began to work. Test FAIL.

SG4860 <--> SG2220 with enabled Async crypto on both sides - no problem with SMB traffic flow. Test OK.

Note: all boxes had AES-NI crypto hw enabled - except 3100! (under Advanced>Misc)

I disabled AES-NI crypto hw on SG2220 and re-tested SG4860 <--> SG2220 enabled Async crypto on both sides - SMB is not being passed through. Disabled async mode - began to work. Test FAIL.

Re-tested SG4860 <--> SG3100 with enabled AES-NI crypto hw on both nodes. Hm, with enabled IPsec Async mode SMB traffic goes through but very slow 355Kb/s and less - it takes a very long time to finalize a humble SMB download. Disabling Async mode on 3100 makes things working.

Also tested SG4860 <--> SG3100 with disabled AES-NI crypto hw on both nodes and enabled IPsec Async mode. Works, but worse then with enabled AES-NI on SG4860 - saw a lot of pauses in SMB traffic transmission.

Trying to summarize what I see:

SG2220 with enabled AES-NI crypto hw AND enabled IPsec Async mode - OK.
SG2220 with enabled AES-NI crypto hw AND disabled IPsec Async mode - OK.
SG2220 with disabled AES-NI crypto hw AND and enabled IPsec Async mode - FAIL.
SG2220 with disabled AES-NI crypto hw AND and disabled IPsec Async mode - OK.

SG3100 with enabled AES-NI crypto hw AND enabled IPsec Async mode - FAIL - very slow bw throughput.
SG3100 with enabled AES-NI crypto hw AND disabled IPsec Async mode - OK.
SG3100 with disabled AES-NI crypto hw AND and enabled IPsec Async mode - FAIL - just not working.
SG3100 with disabled AES-NI crypto hw AND and disabled IPsec Async mode - OK.

*By AES-NI I mean AES-NI and cryptodev

I see the same issues on a SG-1100.

Whoa, SG-1100 is out? Where do I get one?

@luke they’re not for sale yet
@clinton please be more specific
@vladimir please explain how you enabled aes-ni on an arm system. (Do you mean he crypto?)

@Jim I mean "AES-NI and BSD Crypro Device"

System -> Cryptographic:
AES-NI and BSD Crypto Device (aesni, cryptodev)

IPSec -> Advanced Settings -> Asynchronous Cryptography: Enabled

IPSec -> Tunnels:
AES128-GCM (128 bits)
P1 DH-Group: 2 (1024 bit)
No P2 Auth Methods

Running iperf3 across the IPsec tunnel with this topology:
CLIENT -> DUT-A -> DUT-B -> SERVER

Reports a couple Kbps once or twice, then traffic drops completely.

When I disable Async Crypto, I see 75Mbps consistently with PF disabled and 69Mbps with PF enabled (for some reason IPsec GCM is much slower than IPSec CBC).

This is running the latest 2.4.5 snapshot. FYI - The SG-5100 had no issues with this on 2.4.4.

I'm seeing this bug occur on my SG-3100s when using one of the AES-GCM based algorithms for my IPSEC Phase2 with async crypto turned on.

If I switch to plain AES the issue goes away, which happens to use hardware based crypto on the SG-3100s(Which would be a better choice for my SG3100 <--> SG3100 connections probably). I also have SG-3100 <--> intel based pfsense boxes with AES-NI enabled on them. If I just turn off async on the SG-3100 side the problem also goes away. What I haven't tested is trying a non-hardware accelerated algorithm on the intel side with async crypto enabled.

In summary it appears to me that async crypto breaks non-hardware accelerated crypto algorithms or at least the AESXXX-GCM ones.