Bug #13792
openFilterdns assumes sets of resolved addresses for each hostname are nonintersecting
0%
Description
In the current design of filterdns, each hostname thread wake periodically and performs a NS lookup, comparing its current list of mapped addresses to the new addresses returned by the lookup. It then generates add/delete actions for every table the hostname is associated with, and those tables are updated accordingly. This design does not account for the fact that the relation of address -> hostname is not injective, and there will be intersections the sets of addresses associated with hostnames that must be resolved. This can be demonstrated by fabricating NS records for hosts:
a.foo.com (1.1.1.1)
b.foo.com (2.2.2.2)
Then add a.foo.com and b.foo.com entries to a new alias bar , and observe that the bar table is created and populated.
Modify the record for a.foo.com to resolve to 2.2.2.2, causing 2.2.2.2 to have a one-to-many relationship with hostnames a.foo.com and b.foo.com
Wait for the update interval on filterdns to expire
Note in debug logs that address 1.1.1.1 is removed from a.foo.com and 2.2.2.2 is discovered as a new address for a.foo.com
Note that an action is taken for a.foo.com to remove 1.1.1.1 from the table bar for which it is a member
Restore the record for a.foo.com to 1.1.1.1
Wait for the update interval on filterdns to expire
Note in debug logs that address 2.2.2.2 is removed from a.foo.com and 1.1.1.1 is discovered as a new address for a.foo.com
Note that an action is taken for a.foo.com to remove 2.2.2.2 from the table bar for which it is a member
The set of addresses in the table is now incorrect, the address 2.2.2.2 has been removed even though it is still referenced by b.foo.com
To resolve this issue, it is necessary to model the set of addresses associated with each table and maintain a reference count. The creation and deletion of these address associations should drive the add/delete events in the table rather than the set of addresses associated with a hostname.
RL Updated by Reid Linnemann over 3 years ago
- Related to Bug #9296: Alias content is sometimes incomplete when an alias contains both FQDN and IP address entries added
JP Updated by Jim Pingle over 3 years ago
- Plus Target Version changed from 23.05 to 23.09
MM Updated by Marcos M about 3 years ago
- Target version changed from 2.7.0 to CE-Next
- Plus Target Version changed from 23.09 to Plus-Next
JP Updated by Jim Pingle about 3 years ago
- Plus Target Version changed from Plus-Next to 23.09
JP Updated by Jim Pingle about 3 years ago
- Plus Target Version changed from 23.09 to 24.01
JP Updated by Jim Pingle almost 3 years ago
- Plus Target Version changed from 24.01 to 24.03
JP Updated by Jim Pingle over 2 years ago
- Plus Target Version changed from 24.03 to 24.07
JP Updated by Jim Pingle over 2 years ago
- Plus Target Version changed from 24.07 to 24.08
JP Updated by Jim Pingle almost 2 years ago
- Plus Target Version changed from 24.08 to 24.11
JP Updated by Jim Pingle almost 2 years ago
- Plus Target Version changed from 24.11 to 25.01
JP Updated by Jim Pingle almost 2 years ago
- Plus Target Version changed from 25.01 to 25.03
JP Updated by Jim Pingle over 1 year ago
- Plus Target Version changed from 25.03 to 25.07
JM Updated by Javier Martinez over 1 year ago
I have a similar situation here. An alias acting as Whitelist, with two DDNS. When both DDNS are in the same location, they share the same address, so the table is created with only that address. But whenever one of the DDNS moves outside, the older address is deleted and the new one is added, causing the second DDNS to lose access.
DDNS1 ----> 1.2.3.4
DDNS2 ----> 1.2.3.4: Both have access. Table contains 1.2.3.4 only.
1.2.3.4---> DDNS1 ---> 5.6.7.8: Only DDNS1 has access. Table contains 5.6.7.8 only.
1.2.3.4---> DDNS2 ---> 5.5.6.6: Both regain access. Table contains 5.6.7.8 and 5.5.6.6
JP Updated by Jim Pingle about 1 year ago
- Plus Target Version changed from 25.07 to 25.11
SY Updated by Steve Y 10 months ago
I'd point out too that filterdns isn't updating the alias with the valid IP on its next run, it seems to be assuming the valid IP is already in the table. So access (by allow rule) is lost permanently until the IP of that FQDN changes. (or someone runs "killall filterdns" and Status > Filter Reload) If there's not a significant "cost" it could just add each IP at every filterdns update interval, or maybe every "n" runs, as a workaround.
I had linked my thread in one of the duplicate redmine entries so will also link it here: https://forum.netgate.com/topic/196927/filterdns-has-stopped-resolving-hostnames-in-firewall-aliases/
PP Updated by Patch Public 8 months ago
I assume Netgate engineers have already worked out a good way of fixing the various problems with Filterdns. So this is probably not relevant but, this problem was rediscovered and solutions contemplated here https://forum.netgate.com/post/1231744
TL Updated by tmiland labs 1 day ago · Edited
We hit this in production on pfSense 2.8.1: a host alias containing domains with single rotating A records (Cloudflare) loses addresses on every filterdns reload — only the current answer stays in the pf table, so LAN rule traffic sometimes routes via the wrong gateway.
Root-cause analysis confirms filterdns replaces/differs the full address set per hostname each cycle (the non-injective address->hostname assumption noted here). For rotating single-answer DNS this leaves the alias table perpetually incomplete.
Our fix, applied as a System Patch on 2.8.1 and now submitted as a PR to master behind a settings toggle:
- resolve_alias_host_addresses() samples the resolver several times (default 3), follows CNAME chains, and retains previously observed addresses from /var/db/dnscache so the full address set persists across reloads (interleaves with filterdns without waiting for every fqdn entry to poll).
- filter_generate_nested_alias_recurse() resolves hostname entries inline and emits literal "pf <ip> <alias>" filterdns watches for every resolved address.
- New setting "Resolve and retain all FQDN addresses" (System > Advanced > Firewall/NAT, default ON) gates the behavior; off preserves the exact legacy path.
The fix has been submitted upstream as a pull request:
https://github.com/pfsense/pfsense/pull/4763
(PR #4763 — "Add 'Resolve and retain all FQDN addresses' setting for host aliases", links this ticket).
SY Updated by Steve Y about 7 hours ago
That sounds more like a solution for https://docs.netgate.com/pfsense/en/latest/firewall/aliases-features.html#:~:text=round%2Drobin ?
Here the issue is that specific IPs should be included in the alias but are being incorrectly removed because some other hostname changed its IP, even though this one did not. Not that previously-resolved IPs should still be valid.