pfSense

I am (ab)using OpenVPN to extend my network across wireless bridges to mitigate both KRACK and future WPA2 exploits on certain devices until I can run wired ethernet. This also works around the 3 mac address frame limitation in the 802.11 spec that prevents bridging over wireless unless WDS is enabled, which gives 4 mac address frames while halving bandwidth due to side effects. Over the course of setting this up, I have encountered multiple bugs/limitations in pfSense's support for OpenVPN tap devices. Here is a list of the problems that I encountered:

1. Make an OpenVPN server using tap. Try to assign a static IP address to it. The address is not assigned.
2. If you make a DNS server, requests are refused until you configure an access control list.
3. If you get past this, DHCP won't work unless it is assigned to the bridge interface. (maybe not a bug, but it leads to item 4)
4. Changing interface assignments from an existing interface to a parent bridge doesn't work when the original interface is part of the bridge. You must remove the interface from the bridge, then change the assignment and then add it again. If you are configuring pfsense over that interface, you lose access to it. It would be great if these assignments could be done atomically.
5. You cannot put the NTP server on such an interface.
6. You cannot create VLAN interfaces from an OpenVPN tap device.
7. If you create VLAN interfaces from the shell, you cannot bridge them to VLAN interfaces of other adapters. e.g. a vlan from igb and another from from tap.
8. There is no obvious way to simulate AP client isolation when a formerly wireless device is connected this way.

The way that I am extending it is by using LEDE (OpenWRT) on a Linksys EA8500 on the other end. It connects to an AP via WPA2 Enterprise, which assigns a dynamic VLAN as per the radius server. It then connects to the OpenVPN server on pfSense over that. Originally, I wanted to use 802.1q VLANs so that I could have a separate VLAN for each port and bridge, but I hit item 6 and then my attempt to workaround it via item 7 stopped that.

I ended up making an OpenVPN server for each port that I wanted to use. Then I hit issues item 1, item 2 and item 3. While item 3 is a non-issue because you are really supposed to move IP address assignments and bound services to the parent bridge, it lead me to observe item 4, which is a problem because I actually want to do this on the management interface too for something unrelated scenario. To work around that, I made a bridge interface on which the tap interface was added as well as anything else I wanted in the bridge. I made DHCP, DNS and NTP bind to the bridge interface.

For a second device, I wanted an entirely isolated subnet with NTP bound and I wanted information on DHCP leases in status_dhcp_leases.php. Ordinarily, you would go with a tun device for a separate subnet, but it did not seem possible to bind NTP to it (I might be wrong on this). In either case, I wanted information on DHCP leases, so tun was a non-option. I went with tap and hit item 1 and item 5, which meant that I could not bind NTP anyway. I made a bridge interface with the tap device as a sole member to workaround both item 1 and item 5, even though there was really no need for a bridge interface.

Then there is my last issue of item 8, which might just be from my limited understanding of the options. I also encountered issue #8131, but I filed that separately because it has nothing to do with OpenVPN tap devices.

I should add that there were also issues in LEDE that were more critical, so it is not just pfSense that has room for improvement in this area. Although it is not something for the pfSense developers to fix, here is my list:

1. Network connectivity would randomly stop working with no obvious cause until I disabled gso and gro on the bridges and their member interfaces. This was particularly frustrating.
2. The GUI didn't have an option to enable tls-crypt and enabling it required editing configuration files from the shell.
3. I needed to develop custom up and down scripts to make this work.
4. There is no obvious way to simulate AP client isolation when a formerly wireless device is connected this way, much like with pfSense. I likely can fix this with additions to the up/down scripts to use ebtables, so there is a way to do it.

I posted about the setup here:

https://www.reddit.com/r/KRaCK/comments/7f1wxz/workaround_for_devices_that_support_wired_but/