pfSense

I have two firewalls, configured differently as core & edge, both are ver 2.5.0.a.20201006.1250 and I am still seeing thesfollowing issues for the past several updates.

I'm seeing problems with mixed / upper-case alias names and chained alias tables not being parsed as alias names resulting in empty alias tables and broken / non-existent rules and lots of filterdns errors for alias table names.

These were fully functioning chained alias tables, and related rules, that stopped working after the about the time of the PHP 7.4 update; and after a myriad of tracing why traffic was not getting blocked or passed in relation to their aliases, I've found following symptoms occurring:

• When viewing the Firewall -> Aliases page; alias names that are upper or mixed upper-case appear correctly on the left side "Name" column, but the "Values" column of aliases that contain chained aliases will only show aliases in all lower-case. (Similar problem seen in static routing page, however this is not just a cosmetic issue.)
• When editing an alias table that has chained alias name entries that are mixed / upper-case, the initial alias names will be displayed in all lower case. Editing an entry by one backspace to bring up the selection list will show the alias name in correct mixed / upper case. Correct name can be selected and saved, without error, but always appears again as lower case from the alias page, or re-editing the alias.
• Editing alias names to change them, no longer adjust any related alias tables or firewall rules, resulting in configuration rules with broken alias names, and must be manually updated to change the alias name to the new name.
• Alias table names that are mixed upper case and contain only host / network entries are still populated, but can not be used in chained alias tables.
• Alias names that are in all lower case do not seem to be affected, but I've not been able to prove this consistently. Still see areas where editing names may or may not update related chained aliases or rules.
• Looking in /tmp/rules.debug will show alias names with correct mixed / upper-case for both table and variable names, but with no / empty table list.
• From Diagnostics -> Table opening tables with chained aliases, only all lower case chained alias table data is seen. Any data for mixed / upper case alias names is missing. If the table contains only all mixed / upper-case chained aliases, the table will be empty.
• Checking actual rules (pfctl -sr) there are no rules related to chained alias tables where the alias table is empty (Alias name variable points to empty table resulting in invalid source / destination for the given rule, and no errors found relating to empty variables or missing runtime rules).
• DNS Resolver logs will show multitude of filterdns errors for alias names that should be mixed / upper-case as all lower-case, suggesting that the entries are no longer being recognized as alias names and being incorrectly parsed as all lower case host names.
• Excepting filterdns errors, there are no other errors indicating issues regarding blank / empty alias tables or broken firewall rules (due to blank alias tables), other than traffic not not matching their intended rules.

Some digging through the firewall_aliases*.php code it appears that the idn_to_utf8 & idn_to_ascii PHP functions, which are now used in few places when obtaining alias table entries, may be the culprit.

Using the PHP shell with some quick check: echo idn_to_utf8("TeSt\n") results in an all lower-case string 'test'. Same occurs with 'idn_to_ascii' function.

The idn_to_XXX functions were added in issues #7255.

https://redmine.pfsense.org/issues/7255

Looking at the code changes that applied the idn_to_XXX functions to the alias entries, there's no test if an alias entry is an alias table name prior to applying the idn_to_xxx function to the table entry, resulting in table entries being parsed through the idn_to_XXX functions and always lower cased.

https://github.com/pfsense/pfsense/commit/ea12b6073bc5a6ec6ed17325095b685745270d87#diff-f89ae041fab9e90181aab16ebfd1b2d4

The simplest way I've found to reproduce this problem within the pfSense gui is the alias export function, (which now uses the idn_to_utf8 function when mapping the alias array before dumping as a flat txt file). When saving a table that has chained alias names that should be in mixed / upper case, all alias entries are in all lower case.

Other method to reproduce is to simply create a series of alias tables using variations of lower, upper & mixed case and place them in a chained alias table.