Project

General

Profile

Actions

Bug #7841

closed

CARP Sync Issue - when no internet on standby

Added by Brian Stivala over 6 years ago. Updated about 2 years ago.

Status:
Closed
Priority:
Very Low
Assignee:
-
Category:
CARP
Target version:
-
Start date:
09/06/2017
Due date:
% Done:

0%

Estimated time:
Plus Target Version:
Release Notes:
Affected Version:
All
Affected Architecture:
All

Description

Hi all

I've noted a possible bug in pfSense CARP. We have multiple pf instances set up in failover. In some of them, we do not have public IPs available for both primary and the secondary and therefore we are using internal IPs for the WAN interfaces, and a public IP for the CARP.

This works well and fails over normally. However, when making changes on the primary, they often fail with:

Jul 4 09:02:55 php-fpm 32688 /rc.filter_synchronize: XML_RPC_Client: RPC server did not send response before timeout. 103
Jul 4 09:02:55 php-fpm 32688 /rc.filter_synchronize: A communications error occurred while attempting XMLRPC sync with username admin https://172.16.18.2:443.
Jul 4 09:02:55 php-fpm 32688 /rc.filter_synchronize: New alert found: A communications error occurred while attempting XMLRPC sync with username admin https://172.16.18.2:443.

If I reboot the secondary (standby) pfSense, it syncs up all the changes on first boot and then starts throwing that error up around 10 minutes afterwards.

At first, I blamed the internal IPs on the WAN interfaces. Therefore, I replaced the private IPs on one instance and put public IPs throughout. This immediately resolved the issue. So I continued to explore logs to try and identify the actual root cause. As part of this test, I blocked internet access to the secondary (standby) pfSense unit. As soon as I did this, the unit started throwing the above errors.

When using private IPs, the secondary (standby) unit never has internet access until failover occurs. Therefore, this issue seems to be related to the standby unit not having internet and/or not reaching the gateway.

Any ideas?

Actions #1

Updated by Jim Pingle over 6 years ago

  • Assignee deleted (Brian Stivala)
  • Priority changed from High to Very Low
  • Target version changed from 2.3.4-p2 to Future
  • Affected Version changed from 2.3.4 to All

That scenario is rare and not one we technically support or encourage. It's OK for secondary WANs or LANs but both firewalls need external connectivity. This is especially true when you have features enabled which require it. For example, anything that requires DNS resolution such as hostnames in aliases, dynamic DNS, hostnames in VPN peers, aliases which fetch their contents from URLs, packages like pfBlocker which need to download lists, etc.

There may be a way to optimize that in the future but it is not a high priority.

Actions #2

Updated by Yann Tintignac over 6 years ago

Hi Jim,

I had the same issue when using a PfSense cluster with CARP with a /32 Public IP Allocation. I think lot of customers can be impacted by this limitation because not everbody can have a /29 or larger allocation from their ISP.

I believe some specific features such as : WANGW with IP outside the Wan Subnet and CARP / Virtual IP outside the Wan Subnet were implemented to solve the /30->/32 Public allocation on WAN side.

So in this context, I agree with Brian, we should sync the PFSense states / config even when Slave device can not access Internet. This is clearly a situation where the expected setup is Active / Standby. The StandBy node can access Internet only when he becomes Active following a failover.

If this makes sense for you, can you please reconsider the priority of this FR please ?

By advance, thank you.

Actions #3

Updated by Marcos M about 2 years ago

  • Status changed from New to Closed
  • Target version deleted (Future)

Tested on 22.01 following the same steps (blocked secondary node's IP address on upstream firewall). Config sync worked properly and no xmlrpc errors occurred.

Mar 5 15:22:39     php-fpm     99947     /rc.filter_synchronize: Beginning XMLRPC sync data to https://10.254.1.2:443/xmlrpc.php.
Mar 5 15:22:39     php-fpm     99947     /rc.filter_synchronize: XMLRPC reload data success with https://10.254.1.2:443/xmlrpc.php (pfsense.host_firmware_version).
Mar 5 15:22:39     php-fpm     99947     /rc.filter_synchronize: XMLRPC versioncheck: 22.2 -- 22.2
Mar 5 15:22:39     php-fpm     99947     /rc.filter_synchronize: Beginning XMLRPC sync data to https://10.254.1.2:443/xmlrpc.php.
Mar 5 15:22:39     php-fpm     99947     /rc.filter_synchronize: XMLRPC reload data success with https://10.254.1.2:443/xmlrpc.php (pfsense.restore_config_section). 
Actions

Also available in: Atom PDF