Bug #17029
openThreatGate remote feed silently truncated at 4999 entries (26.07)
0%
Description
Netgate 6100, pfSense Plus 26.07-RELEASE.
A ThreatGate remote feed is silently truncated at exactly 4999 entries.
Steps to reproduce:
1. Services > ThreatGate > Remote Feeds > Add Item
2. Mode: Always Block
URL: https://cinsarmy.com/list/ci-badguys.txt
3. Save. Wait for the feed to load.
4. On the firewall shell:
pfctl -t TG_CINS_army -T show | wc -l -> 4999
Expected: 15000 entries (verified on the firewall itself with
fetch -q -o - https://cinsarmy.com/list/ci-badguys.txt | grep -cE "^[0-9]")
Additional findings:
- pf table-entries hard limit is 400000, so that is not the cause.
- Reproduced through both the Nexus GUI (disable/enable + save) and the API.
- The table content is sorted and stops mid-range, confirming truncation
rather than deduplication.
- No error or warning is logged, neither in the GUI nor in
/var/log/pfnet-controller.log
- The same 4999 cap appears on a MaxMind country feed, so it affects
more than one feed source type.
- Splitting the same source into three local feeds of 4900 entries each
loads all 14700, so the cap is per feed rather than global.
The documentation states ThreatGate can "quickly process and utilize huge
lists of addresses" and does not document any per-feed entry limit.
The silent aspect is the main concern: two thirds of a threat intelligence
feed are downloaded, parsed and discarded with no indication to the
administrator that the protection is incomplete.
No data to display
Also available in: Atom