After reinstalling packages at bootup, WebUI not accessable (NanoBSD)
After a firmware update and the first reboot of pfSense installed packages get reinstalled. Sometimes all packages get reinstalled, sometimes only one. (I have Avahi and RRD Summary installed) After the GUI reports that all packages have been reinstalled I can't access the GUI anywhere else. It just says '500 - Internal Server Error'. The firewall is still passing traffic but I need another reboot in order to access the GUI again. Last tested with beta4 built on Sat Nov 27 04:12:08 EST 2010. As a sidenote: Even if only one package like RRD Summary gets reinstalled and Avahi is not listed under 'Installed packages', Avahi is still running 'under the hood' and is started as I can see when connected via console...
Try to keep existing files rather than unlinking/replacing when restoring the package libraries during a package removal. Needs some testing, but for NanoBSD it fixes #1049
#2 Updated by James Lepthien over 8 years ago
Tried updating to latest snap from 11/30 just now. That update broke my box. Boots up says something about an error in some .inc file and then I can see the menu and none of my interfaces has got an ip assigned. I think that kind of problem has been around some time ago...
Where do all the boot messages get written to? You need any logs?
#3 Updated by Jim Pingle over 8 years ago
The commit to fix this just happened. It won't be in a new snapshot until one gets built that includes the fix. It may not be the next one, depending on when the builder started going.
The other error has already been fixed, it should just be a matter of waiting for a new snapshot. No new info is needed right now.
#4 Updated by James Lepthien over 8 years ago
Now tried with snaps from 11/30 and 12/1 and it even got worse. Installed the 12/1 and wanted to connect to the WebUI and this time I even cannot login. So I do not see the reinstalling packages anymore. As soon as I connect to the WebUI I see '500 - Internal Server Error'...
#8 Updated by Jim Pingle over 8 years ago
I just updated an ALIX with cron, OpenVPN client export, shellcmd, blinkled, and siproxd installed. It worked fine. It reinstalled the packages at bootup (watch the console output) and once that was complete I accessed the GUI and saw no errors. I could login and work normally.
Perhaps there is a single common package between those having issues.
#10 Updated by David Szpunar over 8 years ago
I just upgraded from a snapshot from yesterday to the most recent snapshot, full ISO install on a VM, with Open VM Tools and OpenVPN Client Export Utility as the only two installed Packages. After reboot, it loaded the Installing Packages screen and then froze, at the very bottom of the GUI it said:
Fatal error: Call to undefined function pfSense_sync() in /etc/inc/config.lib.inc on line 459
After this I could load nothing and firewall would not route traffic as far as I could tell. Will need to stop onsite tomorrow morning and reload VM from backup and/or reapply config.
I upgraded another box from/to similar snapshot dates and it seems to be working fine. Also a VM, but only Open VM Tools package installed and no others. It upgraded Opem VM Tools with a reinstall as well when I first opened the GUI, but it completed successfully. The locked one above never got past the starting reinstallation screen, no status in the text box showing progress.
#11 Updated by David Szpunar over 8 years ago
I was able to get in remotely (through another firewall/VPN) and revert the VM to a (VM) snapshot from Nov. 3rd that was working. Then I upgraded that to the latest snapshot automatically, and then restored a config backup from yesterday that I had saved, and it seems to be running normally, fully up to date now.
#12 Updated by James Lepthien over 8 years ago
Tried again via console update this time and snap http://snapshots.pfsense.com/FreeBSD_RELENG_8_1/i386/pfSense_HEAD/updates/pfSense-2.0-BETA4-4g-20101202-1127-nanobsd-upgrade.img.gz - on console I could see that all packages but avahi got reinstalled. avahi had been removed, but not reinstalled. So I guess it is this package that brings us this error?
#13 Updated by James Lepthien over 8 years ago
Tried again with snap from 12/5 and my RRD Summary and shellcmd packages got reinstalled, not my avahi and Backup packages though. Today I saw this error on the bottom of the WebUI:
"Warning: fopen(/root/avahi.tar.gz): failed to open stream: Read-only file system in /etc/inc/pfsense-utils.inc on line 1427 Warning: fopen(/usr/local/pkg/avahi.inc): failed to open stream: Read-only file system in /etc/inc/pfsense-utils.inc on line 1427 Warning: fopen(/usr/local/etc/avahi/ssh.service): failed to open stream: Read-only file system in /etc/inc/pfsense-utils.inc on line 1427 Warning: fopen(/usr/local/etc/avahi/sftp-ssh.service): failed to open stream: Read-only file system in /etc/inc/pfsense-utils.inc on line 1427 Warning: fopen(/usr/local/pkg/backup.xml): failed to open stream: Read-only file system in /etc/inc/pfsense-utils.inc on line 1427 Warning: fopen(/usr/local/pkg/backup.inc): failed to open stream: Read-only file system in /etc/inc/pfsense-utils.inc on line 1427 Warning: fopen(/usr/local/www/status_rrd_summary.php): failed to open stream: Read-only file system in /etc/inc/pfsense-utils.inc on line 1427"
Regular procedure then. Reboot and reinstall missing packages by hand...
#16 Updated by Jim Pingle over 8 years ago
- Subject changed from After snapshot update and reinstalling packages WebUI not accessable (NanoBSD) to After reinstalling packages at bootup, WebUI not accessable (NanoBSD)
#17 Updated by Jim Pingle over 8 years ago
- % Done changed from 0 to 70
Should be OK on the next new snapshot dated after this post. (Note that the one building right now does not include the fixes, but the next one after it will!)
Though there may be a couple more things that need investigated next week, but for me it no longer crashes with siproxd or Avahi reinstalling at boot time.
#19 Updated by Jim Pingle over 8 years ago
Keep an eye on it for your next firmware upgrade. Even though there weren't any more changes to that part, I'm curious if -- after reinstalling them this time -- they all come back on the next update.
I did see one update that lost the packages but it was really a holdover from the previous failed update.
#22 Updated by James Lepthien over 8 years ago
Same packages as before. Backup, NMAP, Avahi and RRD Summary. When a package is installed I get the following on the console over and over again:
Mar 22 21:32:16 voldemort kernel: pid 12351 (php), uid 0: exited on signal 11
Mar 22 21:32:16 voldemort kernel: pid 12494 (php), uid 0: exited on signal 11
Mar 22 21:32:16 voldemort kernel: pid 12547 (php), uid 0: exited on signal 11
Mar 22 21:32:16 voldemort kernel: pid 12626 (php), uid 0: exited on signal 11
#23 Updated by Erik Fonnesbeck over 8 years ago
NMap and Avahi appear to cause the error 500 even if installed through the web GUI after the system has fully booted. I'm also getting a segmentation fault when installing RRD Summary during boot. In my opinion, these seem to be problems with the packages themselves, not necessarily individual snapshots.
#24 Updated by Jim Pingle over 8 years ago
- Status changed from New to Resolved
Closing this out. The underlying cause of the original errors have been fixed, and if there are any left they are package specific and tickets should be opened for them individually.
nmap does not generate any errors for me on install, or on firmware upgrade. If avahi has issues, a new ticket with a lot more detail on that would be more helpful at this point.
#25 Updated by Jim Pingle about 7 years ago
- Status changed from Resolved to New
- Target version changed from 2.0 to 2.1
Opening this back up as it's happening again on 2.1.
On my ALIX I have a single package installed, blinkled. After a firmware update, the package reinstalls, and then the GUI is not accessible and there are some signal 11 errors in the system log. Reboot and it's fine.
The blinkled package contains no mount rw/ro calls at all, so it is not something in the individual package's code as before.
With no packages installed the update happens normally and there are no problems getting to the GUI.
#26 Updated by Phillip Davis about 7 years ago
I was trying bandwidthd on a test Alix nanobsd and noticed this also. I didn't report it because bandwidthd doesn't work on 2.1 nanobsd yet anyway (I am looking at fixing up where it writes its output, so it doesn't try to write to the RO CF card).
bandwidthd was the only package I had installed. When I removed bandwidthd, future firmware upgrades booted fine first time.