pfSense

That is odd then. As far as binding, yes and no. I reread the log message, and the complaint is about sendto(), not bind. Notwithstanding that, it still does do bind(), since that is how it causes the source port to be 514. As far as how to repro, I confirmed this on a virtualbox VM. I installed a recent 1.2.3RC3, set it to log to a host on my LAN, and saw the messages. Rebooted, and saw no messages. Looked at /var/log/system.log and see the network unreachable error.

Sounds like syslog fails binding to the local address, as if something else were already bound to 514. What's the exact message? Any packages or anything else non-default installed?

yes, i have a number of packages loaded (squid, havp, etc...) i have been looking at the source on cvsweb and i think i see the bug (not sure why it doesn't always happen). Here is the code after we get an error from sendto:

dprintf("lsent/l: %d/%d\n", lsent, l);
            if (lsent != l) {
                int e = errno;
                logerror("sendto");
                errno = e;
                switch (errno) {
                case ENOBUFS:
                case ENETDOWN:
                case EHOSTUNREACH:
                case EHOSTDOWN:
                    break;
                /* case EBADF: /
                / case EACCES: /
                / case ENOTSOCK: /
                / case EFAULT: /
                / case EMSGSIZE: /
                / case EAGAIN: /
                / case ENOBUFS: /
                / case ECONNREFUSED: */
                default:
                    dprintf("removing entry\n");
                    f->f_type = F_UNUSED;
                    break;
                }

note that ENETDOWN, EHOSTUNREACH and EHOSTDOWN are all non-fatal, whereas the default case is fatal and removes this destination from the list - unfortunately, ENETUNREACHABLE is not listed, and therefore falls into the default case. Unfortunately, if this is wrong, it is in freebsd code, and not anything we can really fix, no? I guess I could work around this, if there was some file like /etc/rc.local in linux, to put customizations?

Hmmm, on the other hand, don't think it package related - my virtualbox repro, is a barebones "quick install" with nothing else done before i repro this.

Definitely something odd here. I booted the VM, and once it was up, from the console, i shutdown the WAN enet (since for the VM, that is how it gets to the syslog server on my LAN), this gets rid of the default gateway, so it cannot now reach 10.0.0.1 (my syslog server). i then did 'ping 10.0.0.1' and saw "Network is down", which is ENETDOWN, which is what syslog treats as transient. It did NOT get the ENETUNREACH.

I can reproduce this in a VM and on a real test box. Adding to the weirdness, from my VM I actually do get log messages through just after the network error (re: tcpdump starting on pflog0) but nothing before or after that point. I get nothing from my test box. Both the VM and test box were updated this morning.

It does start to log again if I send a HUP to syslogd.

I see that Scott adjusted some patches to see if one may have been the cause, hopefully the next snapshot will give us some hints.

This still happens even with a snapshot from Dec 1st, so it doesn't appear the patch changes made a difference.