Bug #1049
closedAfter reinstalling packages at bootup, WebUI not accessable (NanoBSD)
100%
Description
After a firmware update and the first reboot of pfSense installed packages get reinstalled. Sometimes all packages get reinstalled, sometimes only one. (I have Avahi and RRD Summary installed) After the GUI reports that all packages have been reinstalled I can't access the GUI anywhere else. It just says '500 - Internal Server Error'. The firewall is still passing traffic but I need another reboot in order to access the GUI again. Last tested with beta4 built on Sat Nov 27 04:12:08 EST 2010. As a sidenote: Even if only one package like RRD Summary gets reinstalled and Avahi is not listed under 'Installed packages', Avahi is still running 'under the hood' and is started as I can see when connected via console...
Updated by Ermal Luçi over 13 years ago
Can you please try with the latest version of snapshots.
Updated by James Lepthien over 13 years ago
Tried updating to latest snap from 11/30 just now. That update broke my box. Boots up says something about an error in some .inc file and then I can see the menu and none of my interfaces has got an ip assigned. I think that kind of problem has been around some time ago...
Where do all the boot messages get written to? You need any logs?
Updated by Jim Pingle over 13 years ago
The commit to fix this just happened. It won't be in a new snapshot until one gets built that includes the fix. It may not be the next one, depending on when the builder started going.
The other error has already been fixed, it should just be a matter of waiting for a new snapshot. No new info is needed right now.
Updated by James Lepthien over 13 years ago
Now tried with snaps from 11/30 and 12/1 and it even got worse. Installed the 12/1 and wanted to connect to the WebUI and this time I even cannot login. So I do not see the reinstalling packages anymore. As soon as I connect to the WebUI I see '500 - Internal Server Error'...
Updated by Perry Mason over 13 years ago
FYI I've upgrade (with no package) and it worked.
Updated by James Lepthien over 13 years ago
Sure it works without any packages installed. But the ticket is for when you have some packages installed...
Updated by Peter Baumann over 13 years ago
Just want to let you know that the same problem is here too. ALIX Box with latest pfsense 2.0 snapshot.
This problem is actually here since some months with the snapshot releases.
Updated by Jim Pingle over 13 years ago
I just updated an ALIX with cron, OpenVPN client export, shellcmd, blinkled, and siproxd installed. It worked fine. It reinstalled the packages at bootup (watch the console output) and once that was complete I accessed the GUI and saw no errors. I could login and work normally.
Perhaps there is a single common package between those having issues.
Updated by James Lepthien over 13 years ago
I have avahi, rrd summary and shellcmd installed. But I guess it is not shellcmd because I just installed it yesterday. Have this problem with the other two packages since I use the snaps...
Updated by David Szpunar over 13 years ago
I just upgraded from a snapshot from yesterday to the most recent snapshot, full ISO install on a VM, with Open VM Tools and OpenVPN Client Export Utility as the only two installed Packages. After reboot, it loaded the Installing Packages screen and then froze, at the very bottom of the GUI it said:
Fatal error: Call to undefined function pfSense_sync() in /etc/inc/config.lib.inc on line 459
After this I could load nothing and firewall would not route traffic as far as I could tell. Will need to stop onsite tomorrow morning and reload VM from backup and/or reapply config.
I upgraded another box from/to similar snapshot dates and it seems to be working fine. Also a VM, but only Open VM Tools package installed and no others. It upgraded Opem VM Tools with a reinstall as well when I first opened the GUI, but it completed successfully. The locked one above never got past the starting reinstallation screen, no status in the text box showing progress.
Updated by David Szpunar over 13 years ago
I was able to get in remotely (through another firewall/VPN) and revert the VM to a (VM) snapshot from Nov. 3rd that was working. Then I upgraded that to the latest snapshot automatically, and then restored a config backup from yesterday that I had saved, and it seems to be running normally, fully up to date now.
Updated by James Lepthien over 13 years ago
Tried again via console update this time and snap http://snapshots.pfsense.com/FreeBSD_RELENG_8_1/i386/pfSense_HEAD/updates/pfSense-2.0-BETA4-4g-20101202-1127-nanobsd-upgrade.img.gz - on console I could see that all packages but avahi got reinstalled. avahi had been removed, but not reinstalled. So I guess it is this package that brings us this error?
Updated by James Lepthien over 13 years ago
Tried again with snap from 12/5 and my RRD Summary and shellcmd packages got reinstalled, not my avahi and Backup packages though. Today I saw this error on the bottom of the WebUI:
"Warning: fopen(/root/avahi.tar.gz): failed to open stream: Read-only file system in /etc/inc/pfsense-utils.inc on line 1427 Warning: fopen(/usr/local/pkg/avahi.inc): failed to open stream: Read-only file system in /etc/inc/pfsense-utils.inc on line 1427 Warning: fopen(/usr/local/etc/avahi/ssh.service): failed to open stream: Read-only file system in /etc/inc/pfsense-utils.inc on line 1427 Warning: fopen(/usr/local/etc/avahi/sftp-ssh.service): failed to open stream: Read-only file system in /etc/inc/pfsense-utils.inc on line 1427 Warning: fopen(/usr/local/pkg/backup.xml): failed to open stream: Read-only file system in /etc/inc/pfsense-utils.inc on line 1427 Warning: fopen(/usr/local/pkg/backup.inc): failed to open stream: Read-only file system in /etc/inc/pfsense-utils.inc on line 1427 Warning: fopen(/usr/local/www/status_rrd_summary.php): failed to open stream: Read-only file system in /etc/inc/pfsense-utils.inc on line 1427"
Regular procedure then. Reboot and reinstall missing packages by hand...
Updated by James Lepthien over 13 years ago
Sorry. One more thing. After reboot and installing the Backup package I now get a "404 - Not Found" error when trying to navigate...
Updated by Chris Baker over 13 years ago
Just had the problem with the gui not showing when upgrading from 2.0beta4 to beta5. after a reboot it came back up
Updated by Jim Pingle over 13 years ago
- Subject changed from After snapshot update and reinstalling packages WebUI not accessable (NanoBSD) to After reinstalling packages at bootup, WebUI not accessable (NanoBSD)
This is easy to reproduce with siproxd as well, no firmware update required.
Just rm /usr/local/pkg/siproxd* and reboot, and watch it explode.
Here is some debug output and logs:
http://pastie.org/1438178
http://pastie.org/1438185
http://pastie.org/1437465
http://pastie.org/1437495
Updated by Jim Pingle over 13 years ago
- % Done changed from 0 to 70
Should be OK on the next new snapshot dated after this post. (Note that the one building right now does not include the fixes, but the next one after it will!)
Though there may be a couple more things that need investigated next week, but for me it no longer crashes with siproxd or Avahi reinstalling at boot time.
Updated by James Lepthien over 13 years ago
I tested this now with snapshot 1/8 and the problem is kinda gone. Although not all packages got reinstalled automatically, only shellcmd and unbound. I had to manually install avahi, backup and nmap again...
But at least the WebUI bug seems gone.
Updated by Jim Pingle over 13 years ago
Keep an eye on it for your next firmware upgrade. Even though there weren't any more changes to that part, I'm curious if -- after reinstalling them this time -- they all come back on the next update.
I did see one update that lost the packages but it was really a holdover from the previous failed update.
Updated by James Lepthien over 13 years ago
It worked a long time now. Today I updated to the latest snap and now I get the 500 error again, when accessing the WebUI. Needed to reboot...
Updated by Ermal Luçi over 13 years ago
Can you be more verbose:
- What packages you installed?
- Any pertaining log?
Updated by James Lepthien over 13 years ago
Same packages as before. Backup, NMAP, Avahi and RRD Summary. When a package is installed I get the following on the console over and over again:
Mar 22 21:32:16 voldemort kernel: pid 12351 (php), uid 0: exited on signal 11
Mar 22 21:32:16 voldemort kernel: pid 12494 (php), uid 0: exited on signal 11
Mar 22 21:32:16 voldemort kernel: pid 12547 (php), uid 0: exited on signal 11
Mar 22 21:32:16 voldemort kernel: pid 12626 (php), uid 0: exited on signal 11
Updated by Erik Fonnesbeck over 13 years ago
NMap and Avahi appear to cause the error 500 even if installed through the web GUI after the system has fully booted. I'm also getting a segmentation fault when installing RRD Summary during boot. In my opinion, these seem to be problems with the packages themselves, not necessarily individual snapshots.
Updated by Jim Pingle about 13 years ago
- Status changed from New to Resolved
Closing this out. The underlying cause of the original errors have been fixed, and if there are any left they are package specific and tickets should be opened for them individually.
nmap does not generate any errors for me on install, or on firmware upgrade. If avahi has issues, a new ticket with a lot more detail on that would be more helpful at this point.
Updated by Jim Pingle about 12 years ago
- Status changed from Resolved to New
- Target version changed from 2.0 to 2.1
Opening this back up as it's happening again on 2.1.
On my ALIX I have a single package installed, blinkled. After a firmware update, the package reinstalls, and then the GUI is not accessible and there are some signal 11 errors in the system log. Reboot and it's fine.
The blinkled package contains no mount rw/ro calls at all, so it is not something in the individual package's code as before.
With no packages installed the update happens normally and there are no problems getting to the GUI.
Updated by Phillip Davis about 12 years ago
I was trying bandwidthd on a test Alix nanobsd and noticed this also. I didn't report it because bandwidthd doesn't work on 2.1 nanobsd yet anyway (I am looking at fixing up where it writes its output, so it doesn't try to write to the RO CF card).
bandwidthd was the only package I had installed. When I removed bandwidthd, future firmware upgrades booted fine first time.
Updated by Jim Pingle about 12 years ago
- Status changed from New to Feedback
- % Done changed from 70 to 100
Applied in changeset 0b233932f2ec54ddb47a61240fabf02509116767.
Updated by Jim Pingle almost 12 years ago
Applied in changeset 1032ab47c73871993e34446b84767efd235cfd05.
Updated by Jim Pingle almost 12 years ago
- Status changed from Feedback to Resolved
Multiple sources confirm this is fixed.