pfSense

The FreeBSD bug indicates that the bug is triggered by high traffic/bandwidth via the interface.

If someone can recommend to me a way of generating large amounts of traffic between two systems so I can reproduce the bug at-will, I would be happy to confirm and test any bugfixes that you might come up with using my secondary slave pfSense system.

We are seeing this error aswell. We can safely push some 200-300 mbit of traffic, but going beyond that will stop traffic on the interface.
Any update regarding the back port from the 8.2 driver?

Multiple instances of iperf with udp traffic is very good way to generate substantial load.

Can you please try by disabling msix and tso on bge interfaces?

I have had TSO disabled since two weeks ago and have not experienced any crashes, but the systems could go literally months before exhibiting the problem. I'm not sure about how to disable msix or even what it is.

We've got an identical system on the way for a new installation and I was planning to do some testing with generating network traffic to try and reproduce the problem, which I won't bother to do if you plan to give up on this bug.

As mentioned earlier in the ticket, the bug has already been fixed in FreeBSD, just not in the version that PFSense 2.0. We just need someone with some experience and knowledge to help us out with cherry-picking the fix back to the version PFSense is built on to close this bug and the problem users are experiencing on production hardware.

The simplest way to do that is getting a FreeBSD 8.1 box and build the latest bge drivers from FreeBSD HEAD.
Then load the module at boot. It should use that.

The moving of this ticket is not that we give up but with msix disabled and TSO as well disabled systems have behaved so no need to keep this for 2.0

Okay, fair enough. Can you please help me with the process of disabling 'msix'? I'm finding it difficult to discover any resources on it.

hw.pci.enable_msix

See here:
http://doc.pfsense.org/index.php/Tuning_and_Troubleshooting_Network_Cards

Disabling msix on our system caused serious problems (we also have igb Intel Pro cards in these systems which I suspect require this setting).

Symptoms included high CPU load and dropped/hanging connections so this is something that we're not going to be doing on our systems.

Since our broadcom interfaces are driven by the bge driver (and not bce), the setting suggested from the link given by Jim P above does not work. There is no tunable with the name: "hw.bge.tso_enable" so we were unable to disable TSO using this method. I instead used "ifconfig bceX -tso" which I assume does the same thing. We experienced a failure as described in this bug with TSO disabled.

As it appears that this bug is not able to be fixed before the next stable release, and the workarounds either not applicable or with side effects that make them unusable we will simply have to stop using these (onboard) interfaces on these systems and replace them with something else. Luckily our systems have an extra expansion slot which we will install identical Intel igb cards as we already have installed and disable the Broadcom onboard interfaces completely.

I suggest that if others are having issues with these interfaces in high traffic systems that you do the same, especially as this bug occurring will often not trigger fail-over in paired systems and if it does could break network communication on both hosts as it did to us last week.

Also, just as an aside, I was unable to trigger the issue using generated traffic from iperf. I tried to generate both UDP and TCP traffic from single and multiple systems, saturating the 1Gb network connection for up to 30 minutes without experiencing failure. If anybody has any further advice on this I'd be happy to try it.