pfSense

I have a 6rd connection over PPPoE to CenturyLink. I have IPv6 configured on LAN interface to track the WAN, plus 4 (out of 6 total) VLANs are configured to track the WAN. After rebooting, none of the internal interfaces have an IPv6 address assigned.

However, I always have a valid IPv4 and IPv6 address assigned on WAN. I can ping out from pfSense to both IPv4 and IPv6 addresses.

It seems that all I have to do to restore IPv6 connectivity on the internal interfaces is "Save" and "Apply" the WAN interface settings (without making any changes). There seems to be a variety of related problem reports online, none of which seem to have consistent symptoms or solutions. For example:

• https://forum.netgate.com/topic/87871/ipv6-not-working-after-a-reboot-unless-configuration-is-touched
• https://forum.netgate.com/topic/119809/ipv6-6rd-breaks-on-reboot-works-after-release-renew
• https://www.reddit.com/r/PFSENSE/comments/tso9ph/ipv6_not_working_on_startup/

Reviewing my 'system.log', it seems the internal interfaces are configured early, before the PPPoE.

Aug 21 19:14:13 pelenor kernel: lo0: link state changed to UP
Aug 21 19:14:13 pelenor kernel: vmx0: link state changed to UP
Aug 21 19:14:13 pelenor check_reload_status[396]: Linkup starting vmx0
Aug 21 19:14:13 pelenor php[414]: rc.bootup: The interface IPv4 '' address on interface 'wan' is not valid, not configuring 6RD tunnel
Aug 21 19:14:13 pelenor kernel: vlan0: changing name to 'vmx0.2'
Aug 21 19:14:13 pelenor php[414]: rc.bootup: The interface IPv4 '' address on interface 'wan' is not valid, not configuring 6RD tunnel
Aug 21 19:14:13 pelenor kernel: vlan1: changing name to 'vmx0.5'
Aug 21 19:14:13 pelenor php[414]: rc.bootup: The interface IPv4 '' address on interface 'wan' is not valid, not configuring 6RD tunnel
Aug 21 19:14:13 pelenor kernel: vlan2: changing name to 'vmx0.4'
Aug 21 19:14:13 pelenor kernel: vlan3: changing name to 'vmx0.6'
Aug 21 19:14:13 pelenor kernel: vlan4: changing name to 'vmx0.9'
Aug 21 19:14:13 pelenor php[414]: rc.bootup: The interface IPv4 '' address on interface 'wan' is not valid, not configuring 6RD tunnel
Aug 21 19:14:13 pelenor kernel: vlan5: changing name to 'vmx0.7'
Aug 21 19:14:14 pelenor ppp[19863]: Multi-link PPP daemon for FreeBSD
Aug 21 19:14:14 pelenor ppp[19863]:
Aug 21 19:14:14 pelenor ppp[19863]: process 19863 started, version 5.9
Aug 21 19:14:14 pelenor ppp[19863]: web: web is not running
Aug 21 19:14:14 pelenor ppp[19863]: [wan] Bundle: Interface ng0 created
Aug 21 19:14:14 pelenor ppp[19863]: [wan_link0] Link: OPEN event
Aug 21 19:14:14 pelenor kernel:
Aug 21 19:14:14 pelenor kernel: ng0: changing name to 'pppoe0'
Aug 21 19:14:14 pelenor ppp[19863]: [wan_link0] LCP: Open event
Aug 21 19:14:14 pelenor ppp[19863]: [wan_link0] LCP: state change Initial --> Starting
Aug 21 19:14:14 pelenor ppp[19863]: [wan_link0] LCP: LayerStart
Aug 21 19:14:14 pelenor ppp[19863]: [wan_link0] PPPoE: Connecting to 'QuickConnect'
Aug 21 19:14:14 pelenor sshd[20901]: Server listening on :: port 22.
Aug 21 19:14:14 pelenor sshd[20901]: Server listening on 0.0.0.0 port 22.
Aug 21 19:14:14 pelenor sshguard[21485]: Now monitoring attacks.
Aug 21 19:14:16 pelenor check_reload_status[396]: Linkup starting igb0
Aug 21 19:14:16 pelenor kernel: igb0: link state changed to UP
Aug 21 19:14:17 pelenor php[414]: rc.bootup: The interface IPv4 '' address on interface 'pppoe0' is not public, not configuring 6RD tunnel
Aug 21 19:14:17 pelenor php[414]: rc.bootup: The interface IPv4 '' address on interface 'wan' is not valid, not configuring 6RD tunnel
Aug 21 19:14:17 pelenor php[414]: rc.bootup: The interface IPv4 '' address on interface 'wan' is not valid, not configuring 6RD tunnel
Aug 21 19:14:17 pelenor kernel: done.
Aug 21 19:14:17 pelenor php[414]: rc.bootup: The interface IPv4 '' address on interface 'wan' is not valid, not configuring 6RD tunnel
Aug 21 19:14:17 pelenor kernel: done.
Aug 21 19:14:17 pelenor php[414]: rc.bootup: The interface IPv4 '' address on interface 'wan' is not valid, not configuring 6RD tunnel
Aug 21 19:14:17 pelenor kernel: done.
Aug 21 19:14:17 pelenor php[414]: rc.bootup: The interface IPv4 '' address on interface 'wan' is not valid, not configuring 6RD tunnel
Aug 21 19:14:17 pelenor kernel: done.
Aug 21 19:14:17 pelenor php[414]: rc.bootup: Resyncing OpenVPN instances.
Aug 21 19:14:17 pelenor kernel: done.
Aug 21 19:14:17 pelenor kernel: pflog0: promiscuous mode enabled
Aug 21 19:14:17 pelenor kernel: ..
Aug 21 19:14:18 pelenor php[414]: rc.bootup: Gateway, none 'available' for inet, use the first one configured. 'WAN_PPPOE'
Aug 21 19:14:18 pelenor kernel: .done.
Aug 21 19:14:18 pelenor php[414]: rc.bootup: Gateway, none 'available' for inet6, use the first one configured. 'WAN_6RD'
Aug 21 19:14:18 pelenor php[414]: rc.bootup: Default gateway setting Interface WAN_6RD Gateway as default.
Aug 21 19:14:18 pelenor kernel: route: writing to routing socket: Network is unreachable
Aug 21 19:14:18 pelenor kernel:
Aug 21 19:14:18 pelenor kernel: cd0 at ahcich0 bus 0 scbus2 target 0 lun 0
Aug 21 19:14:18 pelenor kernel: cd0: <NECVMWar VMware SATA CD00 1.00> Removable CD-ROM SCSI device
Aug 21 19:14:18 pelenor kernel: cd0: Serial Number 00000000000000000001
Aug 21 19:14:18 pelenor kernel: cd0: 600.000MB/s transfers (SATA 3.x, UDMA2, ATAPI 12bytes, PIO 8192bytes)
Aug 21 19:14:18 pelenor kernel: cd0: Attempt to query device size failed: NOT READY, Medium not present
Aug 21 19:14:18 pelenor php[414]: rc.bootup: sync unbound done.
Aug 21 19:14:18 pelenor kernel: done.
Aug 21 19:14:20 pelenor ppp[19863]: PPPoE: rec'd ACNAME "ptld-dsl-gw51.ptld.qwest.net" 
Aug 21 19:14:20 pelenor ppp[19863]: [wan_link0] PPPoE: connection successful
Aug 21 19:14:20 pelenor ppp[19863]: [wan_link0] Link: UP event
Aug 21 19:14:20 pelenor ppp[19863]: [wan_link0] LCP: Up event
Aug 21 19:14:20 pelenor ppp[19863]: [wan_link0] LCP: state change Starting --> Req-Sent
Aug 21 19:14:20 pelenor ppp[19863]: [wan_link0] LCP: SendConfigReq #1
Aug 21 19:14:20 pelenor ppp[19863]: [wan_link0]   PROTOCOMP
Aug 21 19:14:20 pelenor ppp[19863]: [wan_link0]   MRU 1492
Aug 21 19:14:20 pelenor ppp[19863]: [wan_link0]   MAGICNUM 0xf996cb99
Aug 21 19:14:20 pelenor ppp[19863]: [wan_link0] LCP: rec'd Configure Request #89 (Req-Sent)
Aug 21 19:14:20 pelenor ppp[19863]: [wan_link0]   MRU 1492
Aug 21 19:14:20 pelenor ppp[19863]: [wan_link0]   AUTHPROTO CHAP MD5
Aug 21 19:14:20 pelenor ppp[19863]: [wan_link0]   MAGICNUM 0x642c5e3d
Aug 21 19:14:20 pelenor ppp[19863]: [wan_link0] LCP: SendConfigAck #89
Aug 21 19:14:20 pelenor ppp[19863]: [wan_link0]   MRU 1492
Aug 21 19:14:20 pelenor ppp[19863]: [wan_link0]   AUTHPROTO CHAP MD5
Aug 21 19:14:20 pelenor ppp[19863]: [wan_link0]   MAGICNUM 0x642c5e3d
Aug 21 19:14:20 pelenor ppp[19863]: [wan_link0] LCP: state change Req-Sent --> Ack-Sent
Aug 21 19:14:20 pelenor ppp[19863]: [wan_link0] LCP: rec'd Configure Ack #1 (Ack-Sent)
Aug 21 19:14:20 pelenor ppp[19863]: [wan_link0]   PROTOCOMP
Aug 21 19:14:20 pelenor ppp[19863]: [wan_link0]   MRU 1492
Aug 21 19:14:20 pelenor ppp[19863]: [wan_link0]   MAGICNUM 0xf996cb99
Aug 21 19:14:20 pelenor ppp[19863]: [wan_link0] LCP: state change Ack-Sent --> Opened
Aug 21 19:14:20 pelenor ppp[19863]: [wan_link0] LCP: auth: peer wants CHAP, I want nothing
Aug 21 19:14:20 pelenor ppp[19863]: [wan_link0] LCP: LayerUp
Aug 21 19:14:20 pelenor ppp[19863]: [wan_link0] CHAP: rec'd CHALLENGE #75 len: 27
Aug 21 19:14:20 pelenor ppp[19863]: [wan_link0]   Name: "JUNOS" 
Aug 21 19:14:20 pelenor ppp[19863]: [wan_link0] CHAP: Using authname "endaniel1016@qwest.net" 
Aug 21 19:14:20 pelenor ppp[19863]: [wan_link0] CHAP: sending RESPONSE #75 len: 43
Aug 21 19:14:20 pelenor ppp[19863]: [wan_link0] CHAP: rec'd SUCCESS #75 len: 4
Aug 21 19:14:20 pelenor ppp[19863]: [wan_link0] LCP: authorization successful
Aug 21 19:14:20 pelenor ppp[19863]: [wan_link0] Link: Matched action 'bundle "wan" ""'
Aug 21 19:14:20 pelenor ppp[19863]: [wan_link0] Link: Join bundle "wan" 
Aug 21 19:14:20 pelenor ppp[19863]: [wan] Bundle: Status update: up 1 link, total bandwidth 64000 bps
Aug 21 19:14:20 pelenor ppp[19863]: [wan] IPCP: Open event
Aug 21 19:14:20 pelenor ppp[19863]: [wan] IPCP: state change Initial --> Starting
Aug 21 19:14:20 pelenor ppp[19863]: [wan] IPCP: LayerStart
Aug 21 19:14:20 pelenor ppp[19863]: [wan] IPV6CP: Open event
Aug 21 19:14:20 pelenor ppp[19863]: [wan] IPV6CP: state change Initial --> Starting
Aug 21 19:14:20 pelenor ppp[19863]: [wan] IPV6CP: LayerStart
Aug 21 19:14:20 pelenor ppp[19863]: [wan] IPCP: Up event
Aug 21 19:14:20 pelenor ppp[19863]: [wan] IPCP: state change Starting --> Req-Sent
Aug 21 19:14:20 pelenor ppp[19863]: [wan] IPCP: SendConfigReq #1
Aug 21 19:14:20 pelenor ppp[19863]: [wan]   IPADDR 0.0.0.0
Aug 21 19:14:20 pelenor ppp[19863]: [wan]   COMPPROTO VJCOMP, 16 comp. channels, no comp-cid
Aug 21 19:14:20 pelenor ppp[19863]: [wan]   PRIDNS 0.0.0.0
Aug 21 19:14:20 pelenor ppp[19863]: [wan]   SECDNS 0.0.0.0
Aug 21 19:14:20 pelenor ppp[19863]: [wan] IPV6CP: Up event
Aug 21 19:14:20 pelenor ppp[19863]: [wan] IPV6CP: state change Starting --> Req-Sent
Aug 21 19:14:20 pelenor ppp[19863]: [wan] IPV6CP: SendConfigReq #1
Aug 21 19:14:20 pelenor ppp[19863]: [wan] IPCP: rec'd Configure Request #76 (Req-Sent)
Aug 21 19:14:20 pelenor ppp[19863]: [wan]   IPADDR 207.225.84.51
Aug 21 19:14:20 pelenor ppp[19863]: [wan]     207.225.84.51 is OK
Aug 21 19:14:20 pelenor ppp[19863]: [wan] IPCP: SendConfigAck #76
Aug 21 19:14:20 pelenor ppp[19863]: [wan]   IPADDR 207.225.84.51
Aug 21 19:14:20 pelenor ppp[19863]: [wan] IPCP: state change Req-Sent --> Ack-Sent
Aug 21 19:14:20 pelenor ppp[19863]: [wan] IPCP: rec'd Configure Reject #1 (Ack-Sent)
Aug 21 19:14:20 pelenor ppp[19863]: [wan]   COMPPROTO VJCOMP, 16 comp. channels, no comp-cid
Aug 21 19:14:20 pelenor ppp[19863]: [wan] IPCP: SendConfigReq #2
Aug 21 19:14:20 pelenor ppp[19863]: [wan]   IPADDR 0.0.0.0
Aug 21 19:14:20 pelenor ppp[19863]: [wan]   PRIDNS 0.0.0.0
Aug 21 19:14:20 pelenor ppp[19863]: [wan]   SECDNS 0.0.0.0
Aug 21 19:14:20 pelenor ppp[19863]: [wan_link0] LCP: rec'd Protocol Reject #90 (Opened)
Aug 21 19:14:20 pelenor ppp[19863]: [wan_link0] LCP: protocol IPV6CP was rejected
Aug 21 19:14:20 pelenor ppp[19863]: [wan] IPV6CP: protocol was rejected by peer
Aug 21 19:14:20 pelenor ppp[19863]: [wan] IPV6CP: state change Req-Sent --> Stopped
Aug 21 19:14:20 pelenor ppp[19863]: [wan] IPV6CP: LayerFinish
Aug 21 19:14:20 pelenor ppp[19863]: [wan] IPCP: rec'd Configure Nak #2 (Ack-Sent)
Aug 21 19:14:20 pelenor ppp[19863]: [wan]   IPADDR 97.120.33.24
Aug 21 19:14:20 pelenor ppp[19863]: [wan]     97.120.33.24 is OK
Aug 21 19:14:20 pelenor ppp[19863]: [wan]   PRIDNS 205.171.3.65
Aug 21 19:14:20 pelenor ppp[19863]: [wan]   SECDNS 205.171.2.65
Aug 21 19:14:20 pelenor ppp[19863]: [wan] IPCP: SendConfigReq #3
Aug 21 19:14:20 pelenor ppp[19863]: [wan]   IPADDR 97.120.33.24
Aug 21 19:14:20 pelenor ppp[19863]: [wan]   PRIDNS 205.171.3.65
Aug 21 19:14:20 pelenor ppp[19863]: [wan]   SECDNS 205.171.2.65
Aug 21 19:14:20 pelenor ppp[19863]: [wan] IPCP: rec'd Configure Ack #3 (Ack-Sent)
Aug 21 19:14:20 pelenor ppp[19863]: [wan]   IPADDR 97.120.33.24
Aug 21 19:14:20 pelenor ppp[19863]: [wan]   PRIDNS 205.171.3.65
Aug 21 19:14:20 pelenor ppp[19863]: [wan]   SECDNS 205.171.2.65
Aug 21 19:14:20 pelenor ppp[19863]: [wan] IPCP: state change Ack-Sent --> Opened
Aug 21 19:14:20 pelenor ppp[19863]: [wan] IPCP: LayerUp
Aug 21 19:14:20 pelenor ppp[19863]: [wan]   97.120.33.24 -> 207.225.84.51
Aug 21 19:14:20 pelenor check_reload_status[396]: Rewriting resolv.conf
Aug 21 19:14:21 pelenor check_reload_status[396]: rc.newwanip starting pppoe0
Aug 21 19:14:21 pelenor ppp[19863]: [wan] IFACE: Up event
Aug 21 19:14:21 pelenor ppp[19863]: [wan] IFACE: Rename interface ng0 to pppoe0
Aug 21 19:14:21 pelenor ppp[19863]: [wan] IFACE: Add description "WAN" 
Aug 21 19:14:22 pelenor php-fpm[359]: /rc.newwanip: rc.newwanip: Info: starting on pppoe0.
Aug 21 19:14:22 pelenor php-fpm[359]: /rc.newwanip: rc.newwanip: on (IP address: 97.120.33.24) (interface: WAN[wan]) (real interface: pppoe0).
Aug 21 19:14:22 pelenor php-fpm[359]: /rc.newwanip: Calling interface_6rd_configure(wan, <config>)
Aug 21 19:14:22 pelenor kernel:
Aug 21 19:14:22 pelenor kernel: stf0: changing name to 'wan_stf'
Aug 21 19:14:25 pelenor php-fpm[359]: /rc.newwanip: Default gateway setting Interface WAN_PPPOE Gateway as default.
Aug 21 19:14:25 pelenor php-fpm[359]: /rc.newwanip: Gateway, none 'available' for inet6, use the first one configured. 'WAN_6RD'
Aug 21 19:14:25 pelenor rc.gateway_alarm[17359]: >>> Gateway alarm: WAN_6RD (Addr:2602:cd:ab02:4000:: Alarm:1 RTT:0.000ms RTTsd:0.000ms Loss:100%)
Aug 21 19:14:25 pelenor php-fpm[359]: /rc.newwanip: Default gateway setting Interface WAN_6RD Gateway as default.
Aug 21 19:14:25 pelenor check_reload_status[396]: updating dyndns WAN_6RD
Aug 21 19:14:25 pelenor check_reload_status[396]: Restarting IPsec tunnels
Aug 21 19:14:25 pelenor check_reload_status[396]: Restarting OpenVPN tunnels/interfaces
Aug 21 19:14:25 pelenor check_reload_status[396]: Reloading filter
Aug 21 19:14:25 pelenor php-fpm[359]: /rc.newwanip: IP Address has changed, killing states on former IP Address 71.36.101.130.
Aug 21 19:14:28 pelenor php-fpm[359]: /rc.newwanip: sync unbound done.

I found a condition in interface_6rd_configure() that prevents re-configuration while the system is still booting.

        /* configure dependent interfaces */
        if (!platform_booting()) {
                link_interface_to_track6($interface, "update");
        }

If I remove the platform_booting() condition such that link_interface_to_track6() is called at boot time, then rc.newwanip assigns the internal interface IPv6 addresses:

Aug 21 22:52:05 pelenor php-fpm[359]: /rc.newwanip: rc.newwanip: Info: starting on pppoe0.
Aug 21 22:52:05 pelenor php-fpm[359]: /rc.newwanip: rc.newwanip: on (IP address: 97.120.24.80) (interface: WAN[wan]) (real interface: pppoe0).
Aug 21 22:52:05 pelenor php-fpm[359]: /rc.newwanip: Calling interface_6rd_configure(wan, <config>)
Aug 21 22:52:05 pelenor kernel:
Aug 21 22:52:05 pelenor kernel: stf0: changing name to 'wan_stf'
>> Aug 21 22:52:05 pelenor php-fpm[359]: /rc.newwanip: rd6 lan with ipv6 address 2602:61:7818:5001::1 based on wan ipv4 97.120.24.80
>> Aug 21 22:52:05 pelenor php-fpm[359]: /rc.newwanip: rd6 opt2 with ipv6 address 2602:61:7818:5002::1 based on wan ipv4 97.120.24.80
>> Aug 21 22:52:05 pelenor php-fpm[359]: /rc.newwanip: rd6 opt3 with ipv6 address 2602:61:7818:5005::1 based on wan ipv4 97.120.24.80
>> Aug 21 22:52:05 pelenor php-fpm[359]: /rc.newwanip: rd6 opt6 with ipv6 address 2602:61:7818:5009::1 based on wan ipv4 97.120.24.80
>> Aug 21 22:52:05 pelenor php-fpm[359]: /rc.newwanip: rd6 opt7 with ipv6 address 2602:61:7818:5007::1 based on wan ipv4 97.120.24.80
Aug 21 22:52:08 pelenor php-fpm[359]: /rc.newwanip: Default gateway setting Interface WAN_PPPOE Gateway as default.
Aug 21 22:52:08 pelenor rc.gateway_alarm[12184]: >>> Gateway alarm: WAN_6RD (Addr:2602:cd:ab02:4000:: Alarm:1 RTT:0.000ms RTTsd:0.000ms Loss:100%)
Aug 21 22:52:08 pelenor check_reload_status[396]: updating dyndns WAN_6RD
Aug 21 22:52:08 pelenor check_reload_status[396]: Restarting IPsec tunnels
Aug 21 22:52:08 pelenor check_reload_status[396]: Restarting OpenVPN tunnels/interfaces
Aug 21 22:52:08 pelenor check_reload_status[396]: Reloading filter
Aug 21 22:52:08 pelenor php-fpm[359]: /rc.newwanip: Gateway, none 'available' for inet6, use the first one configured. 'WAN_6RD'
Aug 21 22:52:08 pelenor php-fpm[359]: /rc.newwanip: Default gateway setting Interface WAN_6RD Gateway as default.
Aug 21 22:52:08 pelenor php-fpm[359]: /rc.newwanip: IP Address has changed, killing states on former IP Address 97.120.33.24.
Aug 21 22:52:11 pelenor php-fpm[359]: /rc.newwanip: sync unbound done.

Probably my change is not ideal for all configurations, but hopefully this is a solid data point. I see no downsides in the logs or the system functionality. I'm willing to test variations if a more general solution exists.