pfSense

Note: This was i386 hardware, but I'm not sure if that matters or not.

It's a list of IP addresses, one IP on each line. I just tested it in a new install of 2.1.5, and it works fine there too. The only one that never works is a new install of 2.2.2. I can browse to the URL:

https://192.168.1.1/some_file.txt

And I see the list of IP addresses in my web browser, as expected. This works in all versions of pfSense, and I've tried this every time without fail. I'm 100% certain the problem is NOT the text file. I've even tested various different encodings, like ANSI, UTF-8 without BOM, etc, and the results are exactly the same in every test. Everything works except a new install of 2.2.2.

Is there something else I can look at to provide you with more information to find the cause of this bug? Any logs, or something that might provide more info? I have been testing this bug off and on for a months now, and every variation of the test ends up with the same results. New install fails, everything else works. I don't have a lot of hardware to test on, and my last spare is now in service, which leaves me no extra hardware to test on, so I decided it's time to report the bug, since this is as far as I've been able to get with it.

Let me point out that the "URL Table (IPs)" version of this test does not produce any error messages. Therefore, if it SHOULD be producing an error message, that's another problem. Either way, the fact that the "URL (IPs)" version of this test produces a different result is a good clue that something is wrong with pfSense. If the text file were borked, both the "URL (IPs)" and "URL Table (IPs)" versions of this test should produce the same or similar results, but they do not. Furthermore, since pfSense functions correctly in most test circumstances (older versions and auto-update versions), that's another clue that the problem is likely isolated to new installs of 2.2.2.

Why only new installs of 2.2.2 is affected, I don't know. I would have to do more testing to figure that out, but I don't have any more spare hardware to try it and see what happens. One test I would like to do, but can't at the moment, is allow an auto-update to proceed on 2.1.5 to see if I can get the auto-update version to reproduce the bug somehow. Or, alternatively, fiddle with a lot of new installs until I can get it to function without triggering the bug (I'm not so clear on how to that in this case, though).

Maybe you have some ideas of how I might be able to do further tests to help you isolate the problem? I don't have any spare hardware, but maybe I can try setting up pfSense in a virtual machine or something weird like that. If I had a good idea of what direction I should go in testing that would be productive in helping you, then maybe I could put more effort in finding some extra hardware to test on. Could it be a quirk with my hardware? This problem doesn't strike me as a hardware issue, which is why I suggest it might be possible to reproduce the bug in VM's. Correct me if I'm wrong about that.

Perhaps you could post the results of this:

file /path/to/your/some_file.txt

Guessing it's because we enable certificate validation by default in 2.2.x there, and the default self-signed cert will fail that check. If you switch your GUI to HTTP as a test (System>Advanced), and put in the URL as HTTP, does it work?

I tested an auto-upgrade again before doing the test you suggested. The auto-upgrade sort of failed somehow because the machine never rebooted, and it just dropped to a login prompt that didn't respond to login attempts. The Web GUI did show the upgraded version as 2.2.2, so that's something I guess. Although, the test wasn't completely successful, I was able to check if the aliases still worked, and they did, but I can't be sure it was 2.2.2 aliases, or still 2.1.5, since the reboot didn't occur and the upgrade probably didn't complete. I think pfSense is caching the result of the URL contents from before the upgrade, and that's why a new install doesn't work, but an upgrade does.

Chris Buechler wrote:

Guessing it's because we enable certificate validation by default in 2.2.x there, and the default self-signed cert will fail that check. If you switch your GUI to HTTP as a test (System>Advanced), and put in the URL as HTTP, does it work?

No, it did not work, not at first. It was in the "URL Table (IPs)" version of the configuration, which failed to function after I changed the update frequency to 1 day and then saved it to try to get it reload the URL (for some reason pfSense keeps changing the frequency to 7 days). When I switched it to "URL (IPs)", it worked! When I switched it back to "URL Table (IPs)", it continued to work (cached maybe?). So, it looks like this bug in aliases is triggered by Web GUI HTTPS self-signed certificate configuration.

I switched back to HTTPS, and then tested the URL with 127.0.0.1 instead of 192.168.1.1, with the same bug results. I then tested everything again with 192.168.1.1, and the aliases work again, just like they did with the auto-upgrade scenario. Not sure if any of that is relevant, but noted here since I did the work.

It looks like this bug has been isolated! Thanks for the excellent suggestion to test HTTP versus HTTPS, that was the missing tip I needed to finish this.

the upgrade issue you noted is fixed for 2.2.3, release coming next week. Upgrading to the latest snapshot from snapshots.pfsense.org and re-testing might be useful. I don't recall anything along those lines that's changed though, and the root issue is the cert validation.

Could you provide the contents of your some_file.txt?

Was just wondering if it's specific to your file, or any similar file. If the one you attached suffices to replicate, that's fine, thanks.

Yes, the one I attached is sufficient to replicate this issue. I just tested 2.2.3 and 2.2.4 and they both still have the same problem. I tested on different hardware this time, and I was unable to get any workarounds to function with URL-based aliases.

I have discovered a workaround for this problem. I'm not sure if it's a feature or a bug, so I made a separate report for a feature enhancement to make a form field that is easier to manually enter a lot of data in a quick copy-and-paste operation:

https://redmine.pfsense.org/issues/5663

https://192.168.1.1/firewall_aliases.php?tab=all

In URL you must to use same hostname on self-signed certificate. Change 192.168.1.1 by this hostname. Remember to resolve the hostname to 192.168.1.1 on pfsense.

In Pfsense configuration, verify if option "Verify HTTPS certificates when downloading alias URLs" is unchecked (System / Advanced / Firewall & NAT).