| 1 | Working Cluster in XenServer 7.0 with PV NICs running 2.3.2_1
 | 
  
    | 2 | Upgrade secondary to 2.3.3-DEV then to 2.4.0-DEV
 | 
  
    | 3 | Enter CARP Maintenance mode on primary
 | 
  
    | 4 | Upgrade primary to 2.3.3-DEV
 | 
  
    | 5 | CARP OK
 | 
  
    | 6 | Upgrade to 2.4.0-DEV
 | 
  
    | 7 | CARP OK in BACKUP state
 | 
  
    | 8 | Leave CARP Maintenance mode on primary
 | 
  
    | 9 | 
 | 
  
    | 10 | Nov 4 02:57:27 	php-fpm 	286 	/status_carp.php: The command ‘/sbin/ifconfig xn1 ‘172.25.228.17’/’24’ alias vhid ‘1’’ returned exit code ‘1’, the output was ‘ifconfig: ioctl (SIOCAIFADDR): Protocol not available’
 | 
  
    | 11 | Nov 4 02:57:27 	kernel 		ifa_maintain_loopback_route: deletion failed for interface xn1: 3
 | 
  
    | 12 | Nov 4 02:57:27 	kernel 		ifa_maintain_loopback_route: deletion failed for interface xn1: 3
 | 
  
    | 13 | Nov 4 02:57:27 	kernel 		ifa_maintain_loopback_route: deletion failed for interface xn1: 3
 | 
  
    | 14 | Nov 4 02:57:27 	kernel 		carp: 1@xn1: BACKUP -> INIT (hardware interface up)
 | 
  
    | 15 | Nov 4 02:57:27 	kernel 		xn1: promiscuous mode disabled
 | 
  
    | 16 | Nov 4 02:57:27 	php-fpm 	286 	/status_carp.php: The command ‘/sbin/ifconfig xn0 ‘172.25.236.1’/’24’ alias vhid ‘2’’ returned exit code ‘1’, the output was ‘ifconfig: ioctl (SIOCAIFADDR): Protocol not available’
 | 
  
    | 17 | Nov 4 02:57:27 	kernel 		ifa_maintain_loopback_route: deletion failed for interface xn0: 3
 | 
  
    | 18 | Nov 4 02:57:27 	kernel 		ifa_maintain_loopback_route: deletion failed for interface xn0: 3
 | 
  
    | 19 | Nov 4 02:57:27 	kernel 		ifa_maintain_loopback_route: deletion failed for interface xn0: 3
 | 
  
    | 20 | Nov 4 02:57:27 	kernel 		carp: 2@xn0: BACKUP -> INIT (hardware interface up)
 | 
  
    | 21 | Nov 4 02:57:27 	kernel 		xn0: promiscuous mode disabled
 | 
  
    | 22 | Nov 4 02:57:27 	php-fpm 	286 	/status_carp.php: The command ‘/sbin/ifconfig xn2 ‘172.25.237.1’/’24’ alias vhid ‘3’’ returned exit code ‘1’, the output was ‘ifconfig: ioctl (SIOCAIFADDR): Protocol not available’
 | 
  
    | 23 | Nov 4 02:57:27 	kernel 		ifa_maintain_loopback_route: deletion failed for interface xn2: 3
 | 
  
    | 24 | Nov 4 02:57:27 	kernel 		ifa_maintain_loopback_route: deletion failed for interface xn2: 3
 | 
  
    | 25 | Nov 4 02:57:27 	kernel 		ifa_maintain_loopback_route: deletion failed for interface xn2: 3
 | 
  
    | 26 | Nov 4 02:57:27 	kernel 		carp: 3@xn2: BACKUP -> INIT (hardware interface up)
 | 
  
    | 27 | Nov 4 02:57:27 	kernel 		xn2: promiscuous mode disabled
 | 
  
    | 28 | Nov 4 02:57:27 	check_reload_status 		Syncing firewall
 | 
  
    | 29 | Nov 4 02:57:27 	check_reload_status 		Carp backup event
 | 
  
    | 30 | Nov 4 02:57:27 	check_reload_status 		Carp backup event
 | 
  
    | 31 | Nov 4 02:57:27 	check_reload_status 		Carp backup event
 | 
  
    | 32 | Nov 4 02:57:28 	php-fpm 	285 	/rc.filter_synchronize: Beginning XMLRPC sync data to https://172.25.254.2:443/xmlrpc.php.
 | 
  
    | 33 | Nov 4 02:57:28 	php-fpm 	284 	/rc.carpbackup: HA cluster member “(172.25.228.17@xn1): (WAN)” has resumed CARP state “BACKUP” for vhid 1
 | 
  
    | 34 | Nov 4 02:57:28 	php-fpm 	284 	/rc.carpbackup: HA cluster member “(172.25.237.1@xn2): (OPT1)” has resumed CARP state “BACKUP” for vhid 3
 | 
  
    | 35 | Nov 4 02:57:28 	php-fpm 	286 	/rc.carpbackup: HA cluster member “(172.25.236.1@xn0): (LAN)” has resumed CARP state “BACKUP” for vhid 2
 | 
  
    | 36 | Nov 4 02:57:28 	php-fpm 	285 	/rc.filter_synchronize: XMLRPC reload data success with https://172.25.254.2:443/xmlrpc.php (pfsense.host_firmware_version).
 | 
  
    | 37 | Nov 4 02:57:28 	php-fpm 	285 	/rc.filter_synchronize: XMLRPC versioncheck:15.6 -- 15.6
 | 
  
    | 38 | Nov 4 02:57:28 	php-fpm 	285 	/rc.filter_synchronize: Beginning XMLRPC sync data to https://172.25.254.2:443/xmlrpc.php.
 | 
  
    | 39 | Nov 4 02:57:29 	php-fpm 	285 	/rc.filter_synchronize: XMLRPC reload data success with https://172.25.254.2:443/xmlrpc.php (pfsense.restore_config_section).
 | 
  
    | 40 | Nov 4 02:57:29 	php-fpm 	285 	/rc.filter_synchronize: Beginning XMLRPC sync data to https://172.25.254.2:443/xmlrpc.php.
 | 
  
    | 41 | Nov 4 02:57:32 	php-fpm 	285 	/rc.filter_synchronize: XMLRPC reload data success with https://172.25.254.2:443/xmlrpc.php (pfsense.filter_configure). 
 | 
  
    | 42 | 
 | 
  
    | 43 | Secondary still CARP Master
 | 
  
    | 44 | 
 | 
  
    | 45 | Status > CARP blank status for all VIPS on primary
 | 
  
    | 46 | 
 | 
  
    | 47 | No CARP VIPs on interface:
 | 
  
    | 48 | 
 | 
  
    | 49 | xn0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
 | 
  
    | 50 | 	options=3<RXCSUM,TXCSUM>
 | 
  
    | 51 | 	ether ae:3e:72:48:94:c0
 | 
  
    | 52 | 	inet6 fe80::ac3e:72ff:fe48:94c0%xn0 prefixlen 64 scopeid 0x5
 | 
  
    | 53 | 	inet 172.25.236.2 netmask 0xffffff00 broadcast 172.25.236.255
 | 
  
    | 54 | 	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
 | 
  
    | 55 | 	media: Ethernet manual
 | 
  
    | 56 | 	status: active
 | 
  
    | 57 | xn1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
 | 
  
    | 58 | 	options=3<RXCSUM,TXCSUM>
 | 
  
    | 59 | 	ether f2:92:fa:6a:32:79
 | 
  
    | 60 | 	inet6 fe80::f092:faff:fe6a:3279%xn1 prefixlen 64 scopeid 0x6
 | 
  
    | 61 | 	inet 172.25.228.18 netmask 0xffffff00 broadcast 172.25.228.255
 | 
  
    | 62 | 	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
 | 
  
    | 63 | 	media: Ethernet manual
 | 
  
    | 64 | 	status: active
 | 
  
    | 65 | xn2: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
 | 
  
    | 66 | 	options=3<RXCSUM,TXCSUM>
 | 
  
    | 67 | 	ether 76:e9:15:72:72:39
 | 
  
    | 68 | 	inet6 fe80::74e9:15ff:fe72:7239%xn2 prefixlen 64 scopeid 0x7
 | 
  
    | 69 | 	inet 172.25.237.2 netmask 0xffffff00 broadcast 172.25.237.255
 | 
  
    | 70 | 	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
 | 
  
    | 71 | 	media: Ethernet manual
 | 
  
    | 72 | 	status: active
 | 
  
    | 73 | xn3: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
 | 
  
    | 74 | 	options=3<RXCSUM,TXCSUM>
 | 
  
    | 75 | 	ether 2a:ce:85:9d:aa:49
 | 
  
    | 76 | 	inet6 fe80::28ce:85ff:fe9d:aa49%xn3 prefixlen 64 scopeid 0x8
 | 
  
    | 77 | 	inet 172.25.254.1 netmask 0xfffffffc broadcast 172.25.254.3
 | 
  
    | 78 | 	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
 | 
  
    | 79 | 	media: Ethernet manual
 | 
  
    | 80 | 	status: active
 | 
  
    | 81 | 
 | 
  
    | 82 | Reboot Primary
 | 
  
    | 83 | 
 | 
  
    | 84 | Came up and assumed master on all CARP VIPS. Secondary went to all backup.
 | 
  
    | 85 | 
 | 
  
    | 86 | Enter CARP Maintenance mode on Primary.
 | 
  
    | 87 | 
 | 
  
    | 88 | CARP VIPs vanish. No Status in Status > CARP. No CARP VIPs on interfaces, same syslogs as above.
 | 
  
    | 89 | 
 | 
  
    | 90 | Leave CARP Maintenance mode. VIPs reestablished on interfaces, cluster enters normal MASTER/BACKUP status.
 | 
  
    | 91 | 
 |