pfSense

Configuration:

Multi-WAN with interfaces including:
- "COMCASTBUSINESS" (static IP, interface em0)
- "COMCASTHOME" (DHCP, interface em1.40 (VLAN 40 to cable modem))

Gateway groups are in use; the default gateway group has COMCASTBUSINESS at Tier 1.

Policy based routing specifies the gateway group as the gateway for some LANs, and COMCASTHOME as the policy-based gateway for other LANs.

The static IP of COMCASTBUSINESS is the default route in the kernel routing table. Ping, curl, etc. to the Internet from a shell are routed through this default route, and therefore through the COMCASTBUSINESS interface.
Any attempt to specify the em1.40 interface with curl (e.g. CURLOPT_INTERFACE) results in packets that show the em1.40 DHCP address as the source, but are still sent to the Internet via the default em0 interface in the kernel routing table. All such requests therefore fail: returning packets arrive on em1.40 based on the source address and are never received by curl. This is what causes my dynamic DNS updates to Route53 to fail.

Offending code:

Line 329 in /src/inc/dyndns.class sets the interface to be used for the dynamic DNS outbound request as the covered interface itself (COMCASTHOME in my case)

$this->_dnsRequestIf = get_failover_interface($dnsRequestIf);
if ($this->_dnsVerboseLog) {
    log_error(sprintf(gettext('Dynamic DNS (%1$s): running get_failover_interface for %2$s. found %3$s'), $this->_FQDN, $dnsRequestIf, $this->_dnsRequestIf));
}
$this->_dnsRequestIfIP = get_interface_ip($dnsRequestIf);

The curl_exec at line 1392 attempts to route through the covered interface but is actually routed (by the kernel routing table) through COMCASTBUSINESS but with the source address of COMCASTHOME. The request times out and fails for the reason described above.

The log shows this:

Jun 30 17:23:03     php-fpm     92333     /services_dyndns_edit.php: Curl error occurred: Failed to connect to route53.amazonaws.com port 443: Operation timed out
Jun 30 17:23:03     php-fpm     92333     /services_dyndns_edit.php: Dynamic DNS route53 (home.gilmour.net): _checkStatus() starting. 

It would seem to me that any configuration that tries to perform dynamic DNS on an interface that is not the interface used for the default kernel route would run into this problem.

One idea for a fix would be to emit dynamic DNS updates using the default interface for the firewall itself instead of attempting to use the covered DDNS interface, but I don't know if that would cause other problems with any dynamic DNS providers.