pfSense

A pair of ECMP tunnels was created between a tnsr node and a pfSense node.

BGP peering was established over both paths and proper routes were exchanged.

Floating states must be enabled and were.

The tnsr node used both paths for reply traffic for connections made through it.

pfSense, however, only used one or the other path to initiate connections. I could not find anything that indicated why a particular path was selected over the other one.

The BGP routes looked like this:

[24.03-RELEASE]/root: vtysh -c "show ip bgp" 
BGP table version is 12, local router ID is 172.25.228.21, vrf id 0
Default local pref 100, local AS 64122
Status codes:  s suppressed, d damped, h history, * valid, > best, = multipath,
               i internal, r RIB-failure, S Stale, R Removed
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
Origin codes:  i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found

    Network          Next Hop            Metric LocPrf Weight Path
 *> 172.22.123.0/24  0.0.0.0                  0         32768 i
 *= 172.22.124.0/24  169.254.62.5             0             0 64123 i
 *>                  169.254.62.1             0             0 64123 i
 *= 172.29.96.0/24   169.254.62.5             0             0 64123 i
 *>                  169.254.62.1             0             0 64123 i

Displayed  3 routes and 5 total paths

The pfSense routing table looked like this:


[24.03-RELEASE] /root: netstat -rnfinet
Routing tables

Internet:
Destination        Gateway            Flags     Netif Expire
default            172.25.228.1       UGS      vtnet1
[snip]
172.29.96.0/24     169.254.62.1       UG1      ipsec2
172.29.96.0/24     169.254.62.5       UG1      ipsec3

Tested with iperf3 -R -P 4 -B 172.22.123.1 -c 172.29.96.1 -t 600 to an iperf3 server running upstream of the tnsr node. All outbound connections used VTI2 (169.254.62.5).

pfSense IPsec configuration attached in case there are any questions about that.