Project

General

Profile

Bug #1049

After reinstalling packages at bootup, WebUI not accessable (NanoBSD)

Added by James Lepthien almost 6 years ago. Updated almost 4 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
-
Category:
Web Interface
Target version:
Start date:
11/28/2010
Due date:
% Done:

100%

Affected version:
2.0
Affected Architecture:

Description

After a firmware update and the first reboot of pfSense installed packages get reinstalled. Sometimes all packages get reinstalled, sometimes only one. (I have Avahi and RRD Summary installed) After the GUI reports that all packages have been reinstalled I can't access the GUI anywhere else. It just says '500 - Internal Server Error'. The firewall is still passing traffic but I need another reboot in order to access the GUI again. Last tested with beta4 built on Sat Nov 27 04:12:08 EST 2010. As a sidenote: Even if only one package like RRD Summary gets reinstalled and Avahi is not listed under 'Installed packages', Avahi is still running 'under the hood' and is started as I can see when connected via console...

Associated revisions

Revision 1032ab47
Added by Jim Pingle about 4 years ago

Try to keep existing files rather than unlinking/replacing when restoring the package libraries during a package removal. Needs some testing, but for NanoBSD it fixes #1049

Revision 0b233932
Added by Jim Pingle about 4 years ago

Try to keep existing files rather than unlinking/replacing when restoring the package libraries during a package removal. Needs some testing, but for NanoBSD it fixes #1049

History

#1 Updated by Ermal Luçi over 5 years ago

Can you please try with the latest version of snapshots.

#2 Updated by James Lepthien over 5 years ago

Tried updating to latest snap from 11/30 just now. That update broke my box. Boots up says something about an error in some .inc file and then I can see the menu and none of my interfaces has got an ip assigned. I think that kind of problem has been around some time ago...
Where do all the boot messages get written to? You need any logs?

#3 Updated by Jim Pingle over 5 years ago

The commit to fix this just happened. It won't be in a new snapshot until one gets built that includes the fix. It may not be the next one, depending on when the builder started going.

The other error has already been fixed, it should just be a matter of waiting for a new snapshot. No new info is needed right now.

#4 Updated by James Lepthien over 5 years ago

Now tried with snaps from 11/30 and 12/1 and it even got worse. Installed the 12/1 and wanted to connect to the WebUI and this time I even cannot login. So I do not see the reinstalling packages anymore. As soon as I connect to the WebUI I see '500 - Internal Server Error'...

#5 Updated by Perry Mason over 5 years ago

FYI I've upgrade (with no package) and it worked.

#6 Updated by James Lepthien over 5 years ago

Sure it works without any packages installed. But the ticket is for when you have some packages installed...

#7 Updated by Peter Baumann over 5 years ago

Just want to let you know that the same problem is here too. ALIX Box with latest pfsense 2.0 snapshot.
This problem is actually here since some months with the snapshot releases.

#8 Updated by Jim Pingle over 5 years ago

I just updated an ALIX with cron, OpenVPN client export, shellcmd, blinkled, and siproxd installed. It worked fine. It reinstalled the packages at bootup (watch the console output) and once that was complete I accessed the GUI and saw no errors. I could login and work normally.

Perhaps there is a single common package between those having issues.

#9 Updated by James Lepthien over 5 years ago

I have avahi, rrd summary and shellcmd installed. But I guess it is not shellcmd because I just installed it yesterday. Have this problem with the other two packages since I use the snaps...

#10 Updated by David Szpunar over 5 years ago

I just upgraded from a snapshot from yesterday to the most recent snapshot, full ISO install on a VM, with Open VM Tools and OpenVPN Client Export Utility as the only two installed Packages. After reboot, it loaded the Installing Packages screen and then froze, at the very bottom of the GUI it said:

Fatal error: Call to undefined function pfSense_sync() in /etc/inc/config.lib.inc on line 459

After this I could load nothing and firewall would not route traffic as far as I could tell. Will need to stop onsite tomorrow morning and reload VM from backup and/or reapply config.

I upgraded another box from/to similar snapshot dates and it seems to be working fine. Also a VM, but only Open VM Tools package installed and no others. It upgraded Opem VM Tools with a reinstall as well when I first opened the GUI, but it completed successfully. The locked one above never got past the starting reinstallation screen, no status in the text box showing progress.

#11 Updated by David Szpunar over 5 years ago

I was able to get in remotely (through another firewall/VPN) and revert the VM to a (VM) snapshot from Nov. 3rd that was working. Then I upgraded that to the latest snapshot automatically, and then restored a config backup from yesterday that I had saved, and it seems to be running normally, fully up to date now.

#12 Updated by James Lepthien over 5 years ago

Tried again via console update this time and snap http://snapshots.pfsense.com/FreeBSD_RELENG_8_1/i386/pfSense_HEAD/updates/pfSense-2.0-BETA4-4g-20101202-1127-nanobsd-upgrade.img.gz - on console I could see that all packages but avahi got reinstalled. avahi had been removed, but not reinstalled. So I guess it is this package that brings us this error?

#13 Updated by James Lepthien over 5 years ago

Tried again with snap from 12/5 and my RRD Summary and shellcmd packages got reinstalled, not my avahi and Backup packages though. Today I saw this error on the bottom of the WebUI:

"Warning: fopen(/root/avahi.tar.gz): failed to open stream: Read-only file system in /etc/inc/pfsense-utils.inc on line 1427 Warning: fopen(/usr/local/pkg/avahi.inc): failed to open stream: Read-only file system in /etc/inc/pfsense-utils.inc on line 1427 Warning: fopen(/usr/local/etc/avahi/ssh.service): failed to open stream: Read-only file system in /etc/inc/pfsense-utils.inc on line 1427 Warning: fopen(/usr/local/etc/avahi/sftp-ssh.service): failed to open stream: Read-only file system in /etc/inc/pfsense-utils.inc on line 1427 Warning: fopen(/usr/local/pkg/backup.xml): failed to open stream: Read-only file system in /etc/inc/pfsense-utils.inc on line 1427 Warning: fopen(/usr/local/pkg/backup.inc): failed to open stream: Read-only file system in /etc/inc/pfsense-utils.inc on line 1427 Warning: fopen(/usr/local/www/status_rrd_summary.php): failed to open stream: Read-only file system in /etc/inc/pfsense-utils.inc on line 1427"

Regular procedure then. Reboot and reinstall missing packages by hand...

#14 Updated by James Lepthien over 5 years ago

Sorry. One more thing. After reboot and installing the Backup package I now get a "404 - Not Found" error when trying to navigate...

#15 Updated by Chris Baker over 5 years ago

Just had the problem with the gui not showing when upgrading from 2.0beta4 to beta5. after a reboot it came back up

#16 Updated by Jim Pingle over 5 years ago

  • Subject changed from After snapshot update and reinstalling packages WebUI not accessable (NanoBSD) to After reinstalling packages at bootup, WebUI not accessable (NanoBSD)

This is easy to reproduce with siproxd as well, no firmware update required.

Just rm /usr/local/pkg/siproxd* and reboot, and watch it explode.

Here is some debug output and logs:
http://pastie.org/1438178
http://pastie.org/1438185
http://pastie.org/1437465
http://pastie.org/1437495

#17 Updated by Jim Pingle over 5 years ago

  • % Done changed from 0 to 70

Should be OK on the next new snapshot dated after this post. (Note that the one building right now does not include the fixes, but the next one after it will!)

Though there may be a couple more things that need investigated next week, but for me it no longer crashes with siproxd or Avahi reinstalling at boot time.

#18 Updated by James Lepthien over 5 years ago

I tested this now with snapshot 1/8 and the problem is kinda gone. Although not all packages got reinstalled automatically, only shellcmd and unbound. I had to manually install avahi, backup and nmap again...

But at least the WebUI bug seems gone.

#19 Updated by Jim Pingle over 5 years ago

Keep an eye on it for your next firmware upgrade. Even though there weren't any more changes to that part, I'm curious if -- after reinstalling them this time -- they all come back on the next update.

I did see one update that lost the packages but it was really a holdover from the previous failed update.

#20 Updated by James Lepthien over 5 years ago

It worked a long time now. Today I updated to the latest snap and now I get the 500 error again, when accessing the WebUI. Needed to reboot...

#21 Updated by Ermal Luçi over 5 years ago

Can you be more verbose:
- What packages you installed?
- Any pertaining log?

#22 Updated by James Lepthien over 5 years ago

Same packages as before. Backup, NMAP, Avahi and RRD Summary. When a package is installed I get the following on the console over and over again:

Mar 22 21:32:16 voldemort kernel: pid 12351 (php), uid 0: exited on signal 11
Mar 22 21:32:16 voldemort kernel: pid 12494 (php), uid 0: exited on signal 11
Mar 22 21:32:16 voldemort kernel: pid 12547 (php), uid 0: exited on signal 11
Mar 22 21:32:16 voldemort kernel: pid 12626 (php), uid 0: exited on signal 11

#23 Updated by Erik Fonnesbeck over 5 years ago

NMap and Avahi appear to cause the error 500 even if installed through the web GUI after the system has fully booted. I'm also getting a segmentation fault when installing RRD Summary during boot. In my opinion, these seem to be problems with the packages themselves, not necessarily individual snapshots.

#24 Updated by Jim Pingle over 5 years ago

  • Status changed from New to Resolved

Closing this out. The underlying cause of the original errors have been fixed, and if there are any left they are package specific and tickets should be opened for them individually.

nmap does not generate any errors for me on install, or on firmware upgrade. If avahi has issues, a new ticket with a lot more detail on that would be more helpful at this point.

#25 Updated by Jim Pingle about 4 years ago

  • Status changed from Resolved to New
  • Target version changed from 2.0 to 2.1

Opening this back up as it's happening again on 2.1.

On my ALIX I have a single package installed, blinkled. After a firmware update, the package reinstalls, and then the GUI is not accessible and there are some signal 11 errors in the system log. Reboot and it's fine.

The blinkled package contains no mount rw/ro calls at all, so it is not something in the individual package's code as before.

With no packages installed the update happens normally and there are no problems getting to the GUI.

#26 Updated by Phillip Davis about 4 years ago

I was trying bandwidthd on a test Alix nanobsd and noticed this also. I didn't report it because bandwidthd doesn't work on 2.1 nanobsd yet anyway (I am looking at fixing up where it writes its output, so it doesn't try to write to the RO CF card).
bandwidthd was the only package I had installed. When I removed bandwidthd, future firmware upgrades booted fine first time.

#27 Updated by Jim Pingle about 4 years ago

  • Status changed from New to Feedback
  • % Done changed from 70 to 100

#29 Updated by Jim Pingle almost 4 years ago

  • Status changed from Feedback to Resolved

Multiple sources confirm this is fixed.

Also available in: Atom PDF