Bug #4559
closed
Sync States causes sessions to NOT be NATed with multicast mac
0%
Description
I am using Microsoft NLB for OWA. It uses a multicast MAC address for the cluster, which is fine as long as sync states is disabled.
When Synchronize States is enabled and both firewalls are online the following happens:
Connection establishes correctly, and some traffic passes. Eventually the connection stops NATing traffic out and pfsense sends the reply packet from the OWA server with a source address of it's internal localnet IP (192.168.0.20). This of course does not work properly as that is not allowed to travel back to the client over the internet, and it is not coming from the address the connection was established to in any case.
Using CARP maintenance mode on either firewall does not affect this. The only two cases where traffic continues to work properly is Synchronize States being unchecked or EITHER firewall being offline (reboot etc). Connecting through the same rules to another internal address that is not NLB also corrects this issue.
Yes, net.link.ether.inet.allow_multicast=1 is set on both firewalls.
Updated by Chris Buechler about 9 years ago
you sure it's specific to multicast MACs? Not sure how that would affect it. It sounds like what's happening is the secondary is somehow inserting or deleting a state that's breaking NAT. Layer 2 shouldn't be relevant.
Go to Diag>States, and filter for the IP in question. What do the relevant states look like there?
Updated by Sam Bingner about 9 years ago
To the NLB IP:
WAN tcp 192.168.0.20:443 (64.62.210.7:443) <- 72.234.157.157:13618 ESTABLISHED:ESTABLISHED LAN tcp 72.234.157.157:13618 -> 192.168.0.20:443 ESTABLISHED:ESTABLISHED LAN tcp 192.168.0.20:443 <- 72.234.157.157:13618 ESTABLISHED:ESTABLISHED
ONLY changed NAT destination to be second member in NLB cluster:
WAN tcp 192.168.0.22:443 (64.62.210.7:443) <- 72.234.157.157:57181 ESTABLISHED:ESTABLISHED LAN tcp 72.234.157.157:57181 -> 192.168.0.22:443 ESTABLISHED:ESTABLISHED
Disabled NLB and assigned 0.20 to a single host directly:
WAN tcp 192.168.0.20:443 (64.62.210.7:443) <- 72.234.157.157:9295 ESTABLISHED:ESTABLISHED LAN tcp 72.234.157.157:9295 -> 192.168.0.20:443 ESTABLISHED:ESTABLISHED
I also watched the external interface with tcpdump for any leakage of my internal addresses, and when it was no longer a multicast ARP entry it never happened. Whenever it was, it did.
Updated by Sam Bingner about 9 years ago
Hmm, I found the cause of the problem. The multicast traffic is being flooded to the network and the secondary pfsense instance sees it on the internal interface. It then adds a state that says it's an internal address if the traffic would have been allowed.
I was able to resolve it by changing my LAN ACLs to not allow external addresses from inside. This may be a sufficient fix, but "Bypass firewall rules for traffic on the same interface" didn't fix it, and this was not a problem prior to 2.2
Updated by Chris Buechler about 9 years ago
- Status changed from New to Not a Bug
- Target version deleted (
2.2.2) - Affected Version deleted (
2.2)
Thanks, that explains it.
In the base OS of 2.1.x and earlier versions, the system ignored traffic destined to multicast MACs. The base OS in 2.2x and newer will understand that concept and pass that traffic the same as traffic destined to the system's own MACs if permitted by your ruleset.
"Bypass firewall rules for traffic on the same interface" won't, and isn't designed to, accommodate scenarios like that. It adds pass rules with sloppy state for static routes to work around the problems inherent in strict state keeping and asymmetric routing. It bypasses user-configured rules, not all filtering.
No actual bug here, a change in behavior that might bite some, but it's technically correct now where it wasn't previously and the source of the issue here is an incorrect configuration.
Updated by