Project

General

Profile

Actions
Copy link

Bug #8465

closed

Lost default gateway after recover from failover with CARP VIP and HA

Added by Tom DL7BJ about 6 years ago. Updated over 3 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
Renato Botelho
Category:
High Availability
Target version:
2.4.4-p1
Start date:
04/17/2018
Due date:
% Done:

100%

Estimated time:
Plus Target Version:
Release Notes:
Affected Version:
2.4.3
Affected Architecture:
amd64

Description

Both boxes works with SuperMicro Boards which have two interfaces on board and an additional i350 4 Port network card. HA is on dedicated interfaces, directly connected without switch. All other interfaces are connected to a switch with untagged VLANs for every interface.

WAN Master and Slave - Switch VLAN WAN - ISP
LAN Master and Slave - Switch VLAN LAN - Internal net
DMZ Master and Slave - Switch VLAN DMZ - DMZ
GUEST Master and Slave - Switch VLAN Guest - Guest network
OPT Master and Slave - Switch VLAN OPT - currently not used

Master

WAN Interface: Static IPv4 10.10.75.251/24
Gateway: x.x.x.17

Slave

WAN Interface: Static IPv4 10.10.75.252/24
Gateway: x.x.x.17

The gateway is a public IP address, 62.x.x.17 and "use non local gateway" is set. Outbound NAT is also set (This firewall, WAN Interface, CARP VIP).

External IP

Currently there are 4 static external IPs configured as CARP VIP.

The "master" IP for outgoing traffic is x.x.x.20/29, VHID Group on both 20. The advertising frequency is on master Base = 1 and Skew = 0, on slave Base = 1 and Skew = 100.

The other IPs are for incoming traffic to some webservers and the mailrelay in DMZ.

NAT

There is on both machines Outbound NAT: This Firewall, any source port, any destination, any destination port with NAT Address x.x.x.20

Additional Outbound NAT is configured for some machines, ports and the other CARP VIPs, i.e. outgoing mail is the IP of the MX record and so on.

There is no problem if I switch form master to slave. But back from slave to master the default gateway on master is missing. If I set it in the console or simple save it with a click in the GUI of the master WAN interface or System / Routing / Gateways / Edit without changing something, the default gateway is immediatley set.

I have also done some debugging on console:

a) console on master

- enter persistent CARP maintenance mode on MASTER
- failover to slave, all connections established
- default gw lost on master (netstat r)
 leave persistent CARP maintenance mode on MASTER
- all interfaces and services "green"
- only default gw lost
- route add default 62.x.x.17
- all is up

b) console on master

- ifconfig ibg4 down (WAN interface)
- failover to slave, all connections established
- default gw present on master
- ifconfig ibg4 up
- go back to master as active
- all interfaces and services "green"
- only default gw lost
- route add default 62.x.x.17
- all is up

c) console on master

- sysctl net.inet.carp.demotion=250
- failover to slave, all connections established
- default gw present on master
- sysctl net.inet.carp.demotion=-250
- go back to master as active
- all interfaces and services "green"
- default gw present on master!!!
- all is up

I tried c) several times and pf always switches perfectly between master and slave
without lost of any connection.

If I simulate a lost WAN interface with b) the default gw will be present. The default
gw not lost during failover, but when the Master takes over again.

If I set the Master in maintenance mode a) , the default gw is lost immadiatley.

Why the default gateway will be only restored with c) but not with a) or b)?

Files

Download all files
fixgw.sh.txt (173 Bytes) fixgw.sh.txt fixgw Milad Soltanian, 10/05/2020 03:36 PM
fixgw-pf.png (14.9 KB) fixgw-pf.png poc Milad Soltanian, 10/05/2020 03:36 PM
Actions
Copy link

Also available in: Atom PDF