Bug #14313
open
Unable to create nested URL table aliases
0%
Description
In docs there is a phrase:
"URL table aliases can nest other URL table aliases, and URL aliases can nest other URL aliases."
I'm tested it on 23.01 and on 23.05-DEV and I can't create nested alias with 2 URL table aliases inside:
1. If I tried to create 'Type: Host(s)' alias, I got
"The following input errors were detected:
The alias(es): urltest1 urltest2 cannot be nested because they are not of the same type."
2. If I tried to create 'Type: Network(s)' alias, there was no error but I didn't see this new alias in Diagnostics/Tables
3. If I tried to create 'Type: URL (IPs)' alias, I got
"The following input errors were detected:
A valid URL or alias must be provided. Could not fetch usable data from 'urltest1'.
A valid URL or alias must be provided. Could not fetch usable data from 'urltest2'."
4. If I tried to create 'Type: URL Table (IPs)' alias and add one of these URL Table aliases I already created, I got
"The following input errors were detected:
A valid URL must be provided."
5. If I tried to import aliases, I got no errors but I didn't see this new alias in Diagnostics/Tables
Files
Updated by Jim Pingle over 1 year ago
- Status changed from New to Feedback
- Assignee set to Jim Pingle
It's possible the docs are wrong. The nesting was added not too long ago (~1yr) and I recently updated the docs (#12268) based on info in the request there.
See #1603 and #11863 for info on how the testing of URL type aliases should work.
Looks like nesting them in a Network alias was the intended path based on the code. Aliases will not show up in Diag > Tables until you use them in a rule, so your point #2 seems like it is expected behavior and may be correct.
If that is the case, try it in a rule and see what happens. If that works then the docs can be adjusted to clarify that they should be nested within a Network alias.
Updated by Azamat Khakimyanov over 1 year ago
- Status changed from Feedback to Assigned
Tested on 23.01 and on 23.05-RC (built on Tue May 09 02:36:47 UTC 2023)
I tried to create nested URL table alias by using 'Type: Network' alias and adding into this alias 2 URL table aliases and it failed for all pfSense versions.
I got
There were error(s) loading the rules: /tmp/rules.debug:39: syntax error - The line in question reads [39]: TestNestedUrlAlias = "<TestNestedUrlAlias>"
and in /tmp/rules.debug I saw
table <TestNestedUrlAlias> { 64.6.64.6 # 64.6.65.6 }
TestNestedUrlAlias = "<TestNestedUrlAlias>"
where
- 64.6.64.6 was IP-address from first URL table alias
- 64.6.65.6 was IP-address from second URL table alias
So looks like the reason of this issue is that sharp symbol (#) were added between IP-addresses from these URL table aliases when they were joined together.
When I added only one URL table alias into Network alias and created blocking firewall rule, it was working - traffic was blocked. But weird part was I wasn't able to find any evidence of this traffic being blocked:
- nothing in /Status/System Logs/Firewall
- nothing in /Diagnostics/States
- nothing in pfctl -ss
I saw firewall rule in /tmp/rules.debug and pfctl -sa.
So the only evidence that this firewall rule actually worked I saw in pfctl -sr -vv output
@91 block drop in quick on vtnet1 inet proto tcp from any to <TestNestedUrlAlias:1> flags S/SA label "USER_RULE" label "id:1683634452" ridentifier 1683634452
[ Evaluations: 94 Packets: 7 Bytes: 420 States: 0 ]
[ Inserted: uid 0 pid 91508 State Creations: 0 ]
[ Last Active Time: Tue May 9 15:31:45 2023 ]
Updated by Azamat Khakimyanov over 1 year ago
- File url_table_aliases.png url_table_aliases.png added
- File nested_url_table_aliases.png nested_url_table_aliases.png added
A bit more tests:
1. when I used pfBlockerNG's IP lists
- https://feodotracker.abuse.ch/downloads/ipblocklist_recommended.txt
- https://sslbl.abuse.ch/blacklist/sslipblacklist.txt
and created 2 URL table aliases ('url_table_aliases.png') and in cat /tmp/rules.debug I saw
table <urltest1> persist file "/var/db/aliastables/urltest1.txt"
urltest1 = "<urltest1>"
table <urltest2> persist file "/var/db/aliastables/urltest2.txt"
urltest2 = "<urltest2>"
where both files /var/db/aliastables/urltest1.txt and /var/db/aliastables/urltest2.txt were copies of these pfBlockerNG's IP lists I used
These pfBlockerNG's IP lists have text on top about what these lists are, but in Diagnostics/Tables I saw IPs only. So when these URL table aliases were used, this text were cleaned.
But when I created Network Alias with 2 URL table aliases ('nested_url_table_aliases.png') in I saw
table <TestNestedUrlAlias> { # abuse.ch feodo tracker botnet c2 ip blocklist (recommended) # # terms of use: https://feodotracker.abuse.ch/blocklist/ # # for questions please contact feodotracker [at] abuse.ch # # # dstip # end 180 entries 100.6.31.96 103.109.247.10 103.123.223.171 103.140.174.20 103.141.50.79 103.144.201.56 103.175.16.119 103.233.103.85 103.42.86.42 103.87.128.228 104.248.155.133 104.248.178.90 108.190.115.159 108.32.72.145 109.50.128.59 112.222.83.147 113.11.92.30 12.172.173.82 122.184.143.86 122.186.210.254 125.99.69.178 125.99.76.102 128.199.232.159 129.232.146.250 131.106.168.223 139.226.47.229 14.192.241.76 142.189.121.178 144.202.15.58 144.91.122.94 147.219.4.194 148.64.96.100 151.55.186.41 157.119.85.203 159.65.3.147 161.142.98.36 162.248.14.107 162.33.179.67 171.96.192.178 172.114.160.81 172.115.17.50 172.115.177.204 173.22.114.208 173.88.135.179 174.118.68.176 174.4.89.3 176.142.207.63 178.128.23.9 178.152.124.169 178.175.187.254 183.82.107.190 184.153.132.82 184.176.35.223 184.182.66.109 186.64.67.41 192.99.150.39 196.203.37.215 197.148.17.17 198.199.70.22 198.2.51.242 2.36.64.159 201.244.108.183 202.184.123.13 207.204.111.236 212.112.86.37 213.91.235.146 217.165.234.249 217.44.108.89 24.69.137.232 27.109.19.90 27.99.32.26 31.53.29.198 35.143.97.145 37.14.229.220 40.134.85.217 41.186.88.38 41.227.211.88 41.228.22.180 41.230.168.47 43.243.215.210 45.241.249.37 45.76.1.145 47.149.248.80 47.205.25.170 47.21.51.138 47.34.30.133 49.175.72.7 50.68.186.195 50.68.204.71 58.162.223.233 59.28.84.65 63.140.106.180 63.140.106.181 63.140.106.182 63.140.106.183 63.143.92.99 65.190.242.244 66.181.164.43 66.191.69.18 66.230.104.103 67.10.9.125 67.209.195.198 69.114.91.79 69.133.162.35 69.242.31.249 70.112.206.5 70.118.31.26 70.160.67.203 70.28.50.223 70.29.123.54 70.46.220.114 71.38.155.217 71.78.95.86 72.134.124.16 72.205.104.134 72.80.94.230 73.29.92.128 74.33.196.114 74.92.243.115 75.109.111.89 75.143.236.149 75.98.154.19 75.99.168.194 76.16.49.134 76.170.252.153 76.178.148.107 76.86.31.59 77.86.98.236 78.130.215.67 78.92.133.215 79.167.206.93 79.26.184.19 79.77.142.22 80.12.88.148 80.6.50.34 81.229.117.95 82.127.153.75 82.36.36.76 83.110.74.222 84.108.200.161 84.215.202.8 84.35.26.14 85.104.105.67 85.152.152.46 86.140.160.231 86.176.16.18 86.195.14.72 86.236.114.212 86.244.255.82 86.98.17.95 86.99.48.130 87.202.101.164 87.243.146.59 87.57.13.215 88.126.94.4 88.249.231.161 89.101.97.139 89.114.140.100 89.129.109.27 89.79.229.50 90.104.151.37 90.165.109.4 90.211.192.113 90.70.150.94 90.78.147.141 92.1.170.110 92.154.17.149 92.186.69.229 92.20.204.198 92.239.81.124 92.27.86.48 92.9.45.20 93.147.235.8 94.30.98.134 96.56.197.26 96.87.28.170 98.145.23.67 98.160.111.18 98.19.234.243 99.230.89.236 # abuse.ch sslbl botnet c2 ip blacklist (ips only) # # terms of use: https://sslbl.abuse.ch/blacklist/ # # for questions please contact sslbl [at] abuse.ch # # # dstip # end (31) entries 217.195.197.82 45.204.126.250 45.77.34.211 45.66.230.222 141.98.102.235 84.54.50.51 35.157.111.131 209.90.234.22 194.87.151.125 193.169.255.152 45.137.22.182 15.165.236.45 45.80.158.114 194.87.151.134 31.41.244.251 95.214.27.226 45.141.27.208 87.121.221.179 37.120.210.219 75.127.254.214 120.78.151.171 15.228.89.234 125.177.149.143 194.55.224.44 51.161.107.21 45.81.243.217 61.136.162.141 64.188.16.136 104.219.237.59 23.227.193.141 43.154.97.109 }
TestNestedUrlAlias = "<TestNestedUrlAlias>"
so all text were also added into this Network alias.
Cleaning text script should be added somehow.