pfSense

The reason we have not taken these approaches is primarily because they do not scale. Some people have state tables with hundreds of thousands or millions of entries. PHP code like this will quickly fail with that volume. The processing would have to be handled in a significantly different manner for it to have any chance of working, and would still be slow on affected systems.

If you or someone else can come up with a method that scales well into the millions of states, we'd be interested in having a solution.

I've been working on doing this better and I believe I've come up a solution. I'd appreciate if someone could review these patches to make sure it is all on the right track. My C programming could be rusty and I don't really have the resources to do extensive testing on them, including with the amount of states you'd like to scale to. If I've understood the code correctly, I don't think the performance should be too different from a "pfctl -Fs" call.

The problem I see it is two main parts:
- killing a state would often leave behind a matching state in the opposite direction that is not killed off
- there was no easy way to match states to a gateway (you kill states matched to an interface, but an interface may have multiple gateways)

From reading through the pf code, I've found that a state contains a rt_addr, which seems to correspond to the gateway a state will route through as defined by a policy based routing rule. There is also a function pf_find_state_all, which matches a key (address family, protocol, src & dst addr, src & dst port) and returns a matching state. From what I can tell this uses a hash structure so it should be very quick.

What my kernel patches try to do is:
- add a search parameter to be able to kill states based off the rt_addr
- add a flag which will tell the system to look for and kill a state in the opposite direction based off the specified key and kill it off along with the original state being killed

From there we can patch filter.inc to:
- search for a down gateway and then kill states based off the gateway IP (which will usually be an IN LAN interface state) and also kill matching (OUT WAN interface) states. Killing by gateway here only really applies to states created using policy based routing. Otherwise, states which route using the internal routing table don't define a rt_addr address (more accurately it shows as 0.0.0.0, ::).

The kernel patches also allow us to kill states based of interface and their matching states in the opposite direction with a command like:

 # /sbin/pfctl -M -i interface -Fs

What I'd appreciate feedback or review on is some of the kernel code logic to make sure I have understood it correctly.

In particular:

if (s->direction == PF_OUT) {
  dir = PF_IN;
  idx = PF_SK_STACK;
} else {
  dir = PF_OUT;
  idx = PF_SK_WIRE;
}

Also, I'm not 100% sure about:

match = pf_find_state_all(&key, dir, &more);
if (match && !more) {
  pf_unlink_state(match, 0);
  killed++;
}

Also, as I read it the pf_find_state_all function will not return with the hash row locked, so I am passing 0 as the second argument to pf_unlink_state. If someone could confirm that is correct because almost everywhere else I could find locks the hash row and calls pf_unlink_state with PF_ENTER_LOCKED.

Closing as this is solved by #12092 which ended up covering what is currently possible in this kind of case.