pfSense 2.4.x sending ARP replies with non-CARP source MAC address
pfSense 2.4.x will send ARP replies for CARP interfaces with the local system's "real" source MAC address, instead of using the CARP source address.
Some switches will reject this constellation, and as a consequence, will not add the pfSense gateway MAC address (= CARP address) to their MAC address tables.
Such switches will then flush any traffic destined for the pfSense CARP address to all of its interfaces.
There are multiple reports on the forum about major throughput problems in HA CARP configurations, up to the order of a 90% traffic drop, which I believe may be caused by this problem.
Some switches do not seem to mind about the MAC address mismatch, some switches can be configured to explicitly discard or allow such packets, but some switches just silently drop such mismatched ARP replies and will cause throughput to plummet.
I suspect that some CARP MAC handling code (such as the "net.link.ether.inet.carp_mac" sysctl, which was present in 2.2 and 2.3), hasn't been ported over to pfSense 2.4:
SYSCTL_VNET_INT(_net_link_ether_inet, OID_AUTO, carp_mac, CTLFLAG_RW,
"Send CARP mac with replies to CARP ips");
The patch may have been lost while synching up with FreeBSD, which apparently still doesn't include any mechanism to use the "proper" CARP source address for ARP replies.
I believe earlier pfSense versions even had a different (non-configurable) patch for this problem.
See also: bug #6957