Bug #6001
closed
Not all packages are started during boot time
100%
Description
Hi!
In snapshots from (I think so) 11. march some packages are not started durin boot/reboot.
Example is Lightsquid and Avahi.
See here: https://forum.pfsense.org/index.php?topic=108148.0
When I edited /etc/rc.start_packages and changed "if (time() - $stat['mtime'] >= 90) {" to "if (time() - $stat['mtime'] >= 5) {" problem was resolved.
There is one more thing I noticed...
When all was working just before console shown up, there was text displayed:
Starting package "packagename" for all packages installed.
Now there is no more and I think it was removed somwhere in the proccess.
Funny is that I have 2 affected and 1 not affected systems so it`s not common.
I can provide more logs if needed.
Updated by Chris Buechler over 8 years ago
what's in your /tmp/bootup_messages file?
Updated by Greg M over 8 years ago
I have no such file.
But I can see .rc.start_packages.running file which shouldnt be there right?
Updated by Greg M over 8 years ago
After I manually run rc.start_packages file is created and it has content:
2016-03-15 22:59:48: (network.c.416) can't bind to port: 0.0.0.0 7445 Address already in use
But lightsquid is started and working after this manual intervention.
Also file .rc.start_packages.running is gone.
So it seems that .sh is not running correctly...
Updated by Dmitriy K over 8 years ago
Squid does not start after boot too. Not only Squid.
https://forum.pfsense.org/index.php?topic=108425
Updated by Dmitriy K over 8 years ago
2016/03/17 10:14:00 kid1| /var/run/squid/access.log: (2) No such file or directory
/tmp/cache.log
2016/03/17 10:24:58 kid1| /var/run/squid/squid.pid: (2) No such file or directory
Looks like links issue
Updated by Greg M over 8 years ago
BTW I guess package startup gets interrupted because after boot (like 10 minutes after) there is still /tmp/.rc.start_packages.running file present...
It should be gone but it`s not.
After further testing I think that pppoe might interrupt start process somehow but I`m not sure.
@Dmitriy K:
You have different issue not related to this one.
Updated by Greg M over 8 years ago
And found cause.
So I have snort running on 3 ifaces. It takes a long time to start it up. It starts before lightsquid and because of that lightsquid never starts. I disabled snort on all 3 ifaces, rebooted and lightsquid is up and running.
Updated by Chris Buechler over 8 years ago
- Status changed from New to Not a Bug
- Affected Version deleted (
2.3)
Doesn't appear there is any general problem with package startup (that will be addressed with the status quo of package startup at least). The issue of one package taking ages to startup causing issues for other packages will be addressed by switching to FreeBSD's stock package rc.d in the future. Dmitriy's squid issue is probably something else entirely, please follow up on your forum thread re: that.
Updated by Greg M over 8 years ago
Ok, I understand. But still 10 days ago all was started just fine I don't know what changed, maybe package startup was moved to earlier point in boot time.
Is there a way to change package startup order? So that snort or other heavy packages start last?
EDIT:
Found workaround that works, see: https://forum.pfsense.org/index.php?topic=108148.msg604474#msg604474
Updated by Chad Wagner over 8 years ago
I was not having issues until the same snapshot timeframe as Greg. The workaround Greg mentioned works for Lightsquid but Avahi is still not starting automatically. I am not using snort. Just Squid, Lightsquid, Avahi and openvpn.
Chad
Updated by Jim Pingle over 8 years ago
- Status changed from Not a Bug to New
- Assignee set to Chris Buechler
- Target version set to 2.3
- Affected Version set to 2.3
I finally managed to reproduce this in a 100% reliable fashion. In my test case, WAN is static, WAN2 is DHCP. If WAN2 is enabled, the rc.newwanip call that happens during the boot sequence disrupts the package restart progress, and some services end up not running. If WAN2 is disabled, all of the expected services start.
Chris is digging into it, he's got access to the test system in question.
Updated by Chris Buechler over 8 years ago
- Status changed from New to Confirmed
- Assignee changed from Chris Buechler to Marc Dye
- Priority changed from Normal to High
- Affected Architecture All added
- Affected Architecture deleted (
amd64)
Marc's going to pick up on this. We've discussed it at length.
TLDR version: The issue is when rc.start_packages runs a second time (and exits because it's already running), it somehow stops further processing of this array in the first rc.start_packages instance.
https://github.com/pfsense/pfsense/blob/master/src/etc/rc.start_packages#L65
Updated by Chris Buechler over 8 years ago
- Assignee changed from Marc Dye to Chris Buechler
Updated by Renato Botelho over 8 years ago
- Status changed from Confirmed to Feedback
- Assignee changed from Chris Buechler to Renato Botelho
- % Done changed from 0 to 100
Fixed in check_reload_status 0.0.7
Updated by Chris Buechler over 8 years ago
Looks to be fixed. The 3 cases I had, and one of JimP's, all work fine now. Same root cause likely fixed every other possible combination that was a problem previously.
Others who were seeing issues, working now once you're up to latest?
Updated by Chad Wagner over 8 years ago
All of my services are starting now as well.