Bug #16982
openArpwatch "Update vendors" (ethercodes.dat) silently fails and wipes vendor database
0%
Description
Vendor OUI lookup ("Update vendors" option) silently fails and wipes ethercodes.dat, because arpwatch_update_vendors() uses FreeBSD's fetch command against standards-oui.ieee.org with no failure handling. fetch fails intermittently against that server (about 2 of 3 attempts in testing, "Unknown HTTP error", reproducible over both HTTP and HTTPS), and the existing code pipes the download straight into "> ethercodes.dat", so a failed fetch truncates the file to empty with no error surfaced anywhere. Because arpwatch_update_vendors() runs on every service restart or settings save (via arpwatch_install_command()), not just the monthly update-ethercodes cron job, this leaves the vendor database empty ("unknown" for every entry on Status > Arpwatch > Database) for extended periods in normal use.
Environment
pfSense CE 2.8.1-RELEASE
pfSense-pkg-arpwatch 0.2.3
arpwatch 3.7
Steps to Reproduce
1. Install the Arpwatch.
2. Run from shell:
fetch -qo - https://standards-oui.ieee.org/oui/oui.csv | /usr/local/arpwatch/massagevendor -Z | wc -l
Run it a few times in a row. It frequently returns "fetch: https://standards-oui.ieee.org/oui/oui.csv: Unknown HTTP error" and 0 lines of output, while an equivalent curl-based request succeeds reliably:
/usr/local/bin/curl -s https://standards-oui.ieee.org/oui/oui.csv | /usr/local/arpwatch/massagevendor -Z | wc -l
3. Because the pipe's exit status reflects the last command (massagevendor, which "succeeds" on empty input), arpwatch_update_vendors() in /usr/local/pkg/arpwatch.inc has no way to detect the fetch failure and writes an empty (or truncated) ethercodes.dat regardless.
4. Restart the Arpwatch service, or just save the Settings page (this re-triggers arpwatch_install_command -> arpwatch_update_vendors whenever it happens to hit a fetch failure).
5. Check Status > Arpwatch > Database. The Vendor column shows "unknown" for every entry.
Root Cause
File: /usr/local/pkg/arpwatch.inc, function arpwatch_update_vendors():
function arpwatch_update_vendors($args) {
exec('/usr/bin/fetch -qo - '.ARPWATCH_ETHERCODES_URL.'|'
.ARPWATCH_LOCAL_DIR.'/massagevendor '.$args.' >'
.ARPWATCH_LOCAL_DIR.'/ethercodes.dat');
}
Two problems:
1. fetch seems to be unreliable against standards-oui.ieee.org.
2. The shell redirect (>) truncates ethercodes.dat regardless of whether the pipeline actually produced output, and there is no check on the result before it overwrites the existing (good) file.
The separate, cron-scheduled /usr/local/arpwatch/update-ethercodes script does not have this problem. It uses curl, writes to a temp file first, and only replaces ethercodes.dat if the new download differs from and is non-empty relative to the existing one. arpwatch_update_vendors() doesn't share that safety logic.
Suggested Fix
Use curl instead of fetch and add a temp-file safety check so a failed update doesn't destroy the existing database:
function arpwatch_update_vendors($args) {
$tmp = ARPWATCH_LOCAL_DIR.'/ethercodes.dat.tmp';
exec('/usr/local/bin/curl -s '.ARPWATCH_ETHERCODES_URL.' | '
.ARPWATCH_LOCAL_DIR.'/massagevendor '.$args.' > '.$tmp);
if (file_exists($tmp) && filesize($tmp) > 0) {
rename($tmp, ARPWATCH_LOCAL_DIR.'/ethercodes.dat');
} else {
unlink_if_exists($tmp);
}
}
Updated by Mike A 18 days ago
I can reproduce this on pfSense Plus 26.03.1, with pfSense-pkg-arpwatch 0.2.5 and arpwatch 3.9.
The built-in update-vendors path fails on this system:
[...] 302 redirect to https://standards-oui.ieee.org/oui/oui.csv TLSv1.3 connection established using TLS_AES_128_GCM_SHA256 Peer verification enabled Verify hostname requesting https://standards-oui.ieee.org/oui/oui.csv Request Rejected fetch: http://standards-oui.ieee.org/oui/oui.csv: Unknown HTTP error
The TLS connection and certificate validation succeed; the rejection occurs after the HTTPS request is made.
On the same firewall, from the same WAN address and against the same URL, this succeeds and returns HTTP/1.1 200 OK with the complete CSV:
/usr/local/bin/curl -v -L -o /tmp/oui.csv https://standards-oui.ieee.org/oui/oui.csv
I also reproduced the empty ethercodes.dat result after enabling Update Vendors and saving/reinstalling the package. I have disabled the built in vendor updater and am using a cron curl based update that writes a temporary file, verifies it is non empty, and only then replaces ethercodes.dat.
This confirms the issue on Plus as well as CE.
Also available in: Atom