Project

General

Profile

Bug #10968

Mixed & Upper case Alias table names broken.

Added by Troy Emmerson about 2 months ago. Updated about 1 month ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Aliases / Tables
Target version:
Start date:
10/08/2020
Due date:
% Done:

0%

Estimated time:
Affected Version:
2.5.0
Affected Architecture:
All

Description

I have two firewalls, configured differently as core & edge, both are ver 2.5.0.a.20201006.1250 and I am still seeing thesfollowing issues for the past several updates.

I'm seeing problems with mixed / upper-case alias names and chained alias tables not being parsed as alias names resulting in empty alias tables and broken / non-existent rules and lots of filterdns errors for alias table names.

These were fully functioning chained alias tables, and related rules, that stopped working after the about the time of the PHP 7.4 update; and after a myriad of tracing why traffic was not getting blocked or passed in relation to their aliases, I've found following symptoms occurring:

  • When viewing the Firewall -> Aliases page; alias names that are upper or mixed upper-case appear correctly on the left side "Name" column, but the "Values" column of aliases that contain chained aliases will only show aliases in all lower-case. (Similar problem seen in static routing page, however this is not just a cosmetic issue.)
  • When editing an alias table that has chained alias name entries that are mixed / upper-case, the initial alias names will be displayed in all lower case. Editing an entry by one backspace to bring up the selection list will show the alias name in correct mixed / upper case. Correct name can be selected and saved, without error, but always appears again as lower case from the alias page, or re-editing the alias.
  • Editing alias names to change them, no longer adjust any related alias tables or firewall rules, resulting in configuration rules with broken alias names, and must be manually updated to change the alias name to the new name.
  • Alias table names that are mixed upper case and contain only host / network entries are still populated, but can not be used in chained alias tables.
  • Alias names that are in all lower case do not seem to be affected, but I've not been able to prove this consistently. Still see areas where editing names may or may not update related chained aliases or rules.
  • Looking in /tmp/rules.debug will show alias names with correct mixed / upper-case for both table and variable names, but with no / empty table list.
  • From Diagnostics -> Table opening tables with chained aliases, only all lower case chained alias table data is seen. Any data for mixed / upper case alias names is missing. If the table contains only all mixed / upper-case chained aliases, the table will be empty.
  • Checking actual rules (pfctl -sr) there are no rules related to chained alias tables where the alias table is empty (Alias name variable points to empty table resulting in invalid source / destination for the given rule, and no errors found relating to empty variables or missing runtime rules).
  • DNS Resolver logs will show multitude of filterdns errors for alias names that should be mixed / upper-case as all lower-case, suggesting that the entries are no longer being recognized as alias names and being incorrectly parsed as all lower case host names.
  • Excepting filterdns errors, there are no other errors indicating issues regarding blank / empty alias tables or broken firewall rules (due to blank alias tables), other than traffic not not matching their intended rules.

Some digging through the firewall_aliases*.php code it appears that the idn_to_utf8 & idn_to_ascii PHP functions, which are now used in few places when obtaining alias table entries, may be the culprit.

Using the PHP shell with some quick check: echo idn_to_utf8("TeSt\n") results in an all lower-case string 'test'. Same occurs with 'idn_to_ascii' function.

The idn_to_XXX functions were added in issues #7255.

https://redmine.pfsense.org/issues/7255

Looking at the code changes that applied the idn_to_XXX functions to the alias entries, there's no test if an alias entry is an alias table name prior to applying the idn_to_xxx function to the table entry, resulting in table entries being parsed through the idn_to_XXX functions and always lower cased.

https://github.com/pfsense/pfsense/commit/ea12b6073bc5a6ec6ed17325095b685745270d87#diff-f89ae041fab9e90181aab16ebfd1b2d4

The simplest way I've found to reproduce this problem within the pfSense gui is the alias export function, (which now uses the idn_to_utf8 function when mapping the alias array before dumping as a flat txt file). When saving a table that has chained alias names that should be in mixed / upper case, all alias entries are in all lower case.

Other method to reproduce is to simply create a series of alias tables using variations of lower, upper & mixed case and place them in a chained alias table.

Associated revisions

Revision bb3fc251 (diff)
Added by Viktor Gurov about 2 months ago

Nested alias IDN fix. Issue #10968

Revision c91be02b (diff)
Added by Viktor Gurov about 2 months ago

Nested alias IDN fix. Issue #10968

History

#1 Updated by Viktor Gurov about 2 months ago

  • Category set to Aliases / Tables
  • Target version set to 2.5.0

#3 Updated by Renato Botelho about 2 months ago

  • Assignee set to Viktor Gurov

Viktor already have a patch to fix this one

#4 Updated by Renato Botelho about 2 months ago

  • Status changed from New to Feedback

PR has been merged. Thanks!

#5 Updated by Steve Beaver about 2 months ago

  • Assignee changed from Viktor Gurov to Troy Emmerson

#6 Updated by Danilo Zrenjanin about 1 month ago

  • Status changed from Feedback to Resolved

Tested on:

2.5.0-DEVELOPMENT (amd64)
built on Sat Oct 24 01:00:29 EDT 2020
FreeBSD 12.2-STABLE

It works fine. Ticket resolved.

Also available in: Atom PDF