Project

General

Profile

Actions
Copy link

Feature #3013

closed

Better upgrading for a CARP cluster

Added by Jerome Alet almost 11 years ago. Updated over 6 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
-
Category:
CARP
Target version:
2.3
Start date:
05/29/2013
Due date:
% Done:

0%

Estimated time:
Plus Target Version:
Release Notes:

Description

Not sure if this is a bug or something we do incorrectly, so I've added this issue as "Feature".

While upgrading a 2 nodes CARP cluster, we usually upgrade the slave first, and then if all goes well after reboot and packages reinstallation we upgrade the master. The problem is that once the master reboots, all its network interfaces automatically go to CARP MASTER state even before packages begin to be reinstalled, causing some functionnalities which were handled by the slave during the master's reboot to not work anymore, possibly for a long time if there is a lot of packages to download and reinstall.

Couldn't the master be artificially kept in CARP BACKUP mode until all packages have been reinstalled and started ?

Actions
Copy link
 #1

Updated by Chris Buechler almost 11 years ago

  • Target version deleted (2.1)
  • Affected Version deleted (2.1)
Actions
Copy link
 #2

Updated by Chris Buechler about 8 years ago

  • Status changed from New to Resolved
  • Target version set to 2.3

this is fixed in 2.3 by the nature of how upgrades work now, all the packages are downloaded pre-reboot, and are upgraded before the box is brought up on the network.

Actions
Copy link
 #3

Updated by Stéphane Lapie over 6 years ago

I have seen this improvement, so first of all, thank you so much.

However, the problem still remains that the CARP master claims back the IP address before all services are up again.
In my case, I am using pfSense to provide service via HAproxy, and this leaves a time frame where service is not available because :
  1. pfSense master boots
  2. Network is down, and in the meantime packages get installed, which is much better. :)
  3. However, network is brought up and CARP along with it, but HAproxy does not start (for some reason on the next reboot after the upgrade, some services do not get brought up, I have to reboot the instance once again for everything to come back nicely ; but the point remains, there is an ever so slight time window where network is up but userland services are not up yet!)
  4. When rebooting, processes are shut down first but CARP is still up, so I technically have downtime. It might be nice to have CARP be brought down first.
I work around it :
  • By disabling the front-facing network interface after everything has been downloaded, and just before rebooting, but this is not exactly a viable option.
  • By deactivating CARP manually with ifconfig before doing a reboot

An option allowing CARP activation to wait until such and such userland service might be nice,
and a hook to deactivate CARP first when using interface triggered reboots.

Actions
Copy link

Also available in: Atom PDF