pfSense

Our IP block and router status output is aleady attached to the support ticket. I will also attached the other info to the support ticket.

Requested info has been added to the support ticket.

It seems like the IPs are assigned to the interface in ascending order no matter what pppoe server gives

(reverse order)

Mar 5 14:23:49     ppp     62519     [wan] IFACE: Rename interface ng0 to pppoe2
Mar 5 14:23:49     ppp     62519     [wan] IFACE: Up event
Mar 5 14:23:49     ppp     62519     [wan] 1.2.3.222 -> 6.7.8.23
Mar 5 14:23:49     ppp     62519     [wan] IPCP: LayerUp
Mar 5 14:23:49     ppp     62519     [wan] IPCP: state change Ack-Sent --> Opened
Mar 5 14:23:49     ppp     62519     [wan] IPADDR 1.2.3.222
Mar 5 14:23:49     ppp     62519     [wan] IPCP: rec'd Configure Ack #3 (Ack-Sent)
Mar 5 14:23:49     ppp     62519     [wan] IPADDR 1.2.3.222
Mar 5 14:23:49     ppp     62519     [wan] IPCP: SendConfigReq #3
Mar 5 14:23:49     ppp     62519     [wan] 1.2.3.222 is OK
Mar 5 14:23:49     ppp     62519     [wan] IPADDR 1.2.3.222

pppoe2: flags=88d1<UP,POINTOPOINT,RUNNING,NOARP,SIMPLEX,MULTICAST> metric 0 mtu 1492
 description: WAN
 inet6 fe80::2e0:edff:febe:ef64%pppoe2 prefixlen 64 scopeid 0xe
 inet 1.2.3.217 --> 6.7.8.24 netmask 0xffffffff
 inet 1.2.3.218 --> 6.7.8.24 netmask 0xffffffff
 inet 1.2.3.219 --> 6.7.8.24 netmask 0xffffffff
 inet 1.2.3.220 --> 6.7.8.24 netmask 0xffffffff
 inet 1.2.3.221 --> 6.7.8.24 netmask 0xffffffff
 inet 1.2.3.222 --> 6.7.8.24 netmask 0xffffffff
 nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>

Perhaps this is another variation of #11545 and not a unique issue

Jim Pingle wrote:

Perhaps this is another variation of #11545 and not a unique issue

This could be an another issue - if the PPPoE Server changes the IP address pool and `/usr/local/sbin/ppp-linkup` doesn't start, you'll see the updated IP address at the bottom.

Jim Pingle wrote:

We will need a lot more information here since it isn't happening to others that we're aware of yet.

We can confirm we have the same issue. After a ppp reconnect (seems to happen once a week, I guess triggered by the ISP) the wrong IP is assigned here the .76 instead of the .73 (First IP in our public IP block)

We do have 2 virtual IP's defined .76 and .77. The selected IP seems random as it's not the last or lowest or highest in our range and virtual ip list...

Are there any possible workarounds? At the moment this is a big show stopper....
Could we prevent the rc.newwanip from setting a new IP that is the wrong one? We end up with no WAN and the wrong IP on IPSEC and OpenVPN services...

If it would help we can share more logging information...

May 28 01:54:41 php-fpm 54474 /rc.newwanip: Forcefully reloading IPsec
May 28 01:54:38 php-fpm 54474 /rc.newwanip: Default gateway setting Interface WAN_PPPOE Gateway as default.
May 28 01:54:37 php-fpm 54474 /rc.newwanip: rc.newwanip: on (IP address: xx.xx.xxx.76) (interface: WAN[wan]) (real interface: pppoe0).
May 28 01:54:37 php-fpm 54474 /rc.newwanip: rc.newwanip: Info: starting on pppoe0.
May 28 01:54:36 ppp 14153 [wan] IFACE: Rename interface ng0 to pppoe0
May 28 01:54:36 ppp 14153 [wan] IFACE: Up event
May 28 01:54:36 check_reload_status 390 rc.newwanip starting pppoe0
May 28 01:54:36 ppp 14153 [wan] xx.xx.xxx.73 > xxx.x.xxx.196
May 28 01:54:36 ppp 14153 [wan] IPCP: LayerUp
May 28 01:54:36 ppp 14153 [wan] IPCP: state change Ack-Sent -> Opened
May 28 01:54:36 ppp 14153 [wan] IPADDR xx.xx.xxx.73
May 28 01:54:36 ppp 14153 [wan] IPCP: rec'd Configure Ack #11 (Ack-Sent)
May 28 01:54:36 ppp 14153 [wan] IPADDR xx.xx.xxx.73
May 28 01:54:36 ppp 14153 [wan] IPCP: SendConfigReq #11
May 28 01:54:36 ppp 14153 [wan] xx.xx.xxx.73 is OK
May 28 01:54:36 ppp 14153 [wan] IPADDR xx.xx.xxx.73
May 28 01:54:36 ppp 14153 [wan] IPCP: rec'd Configure Nak #10 (Ack-Sent)
May 28 01:54:36 ppp 14153 [wan] IPADDR 0.0.0.0
May 28 01:54:36 ppp 14153 [wan] IPCP: SendConfigReq #10

workaround:

You could use VIPs from your /29 for all the VPNs/services. If clients are using an FQDN you could just update the DNS to point at the VIP.

If that's not possible as a workaround try adding xx.xx.xxx.73 as an IP Alias. It will then be selectable as the server IP directly rather than 'WAN'.

We are experiencing a similar issue on version 2.5.2.
When the ppp connection comes back up after an isp outage rc.newwanip sets the wrong ip address on the wan interface.
A virtual ip from an additional /30 is detected instead of the actual IPADDR.

x.x.x.213: ppp ip 'static' address.
z.z.z.z: ppp gateway
y.y.y.97: a virtual ip address (ip alias on wan interface) from additional /30

May  3 22:47:42 gw ppp[14390]: [wan_link0] LCP: SendTerminateAck #2
May  3 22:47:42 gw ppp[14390]: [wan_link0] LCP: LayerDown
May  3 22:47:42 gw ppp[14390]: [wan_link0] PPPoE: connection closed
May  3 22:47:42 gw ppp[14390]: [wan_link0] Link: DOWN event
May  3 22:47:42 gw ppp[14390]: [wan_link0] LCP: Down event
...
May  4 03:06:21 gw ppp[14390]: [wan] IPCP: rec'd Configure Ack #7 (Ack-Sent)
May  4 03:06:21 gw ppp[14390]: [wan]   IPADDR x.x.x.213
May  4 03:06:21 gw ppp[14390]: [wan] IPCP: state change Ack-Sent --> Opened
May  4 03:06:21 gw ppp[14390]: [wan] IPCP: LayerUp
May  4 03:06:21 gw ppp[14390]: [wan]   x.x.x.213 -> z.z.z.z
May  4 03:06:21 gw check_reload_status[377]: rc.newwanip starting pppoe0
May  4 03:06:21 gw ppp[14390]: [wan] IFACE: Up event
May  4 03:06:21 gw ppp[14390]: [wan] IFACE: Rename interface ng0 to pppoe0
May  4 03:06:23 gw php-fpm[38147]: /rc.newwanip: rc.newwanip: Info: starting on pppoe0.
May  4 03:06:23 gw php-fpm[38147]: /rc.newwanip: rc.newwanip: on (IP address: y.y.y.97) (interface: WAN[wan]) (real interface: pppoe0).
May  4 03:06:24 gw php-fpm[38147]: /rc.newwanip: Default gateway setting Interface WAN_PPPOE Gateway as default.
May  4 03:06:24 gw php-fpm[38147]: /rc.newwanip: Gateway, none 'available' for inet6, use the first one configured. ''
May  4 03:06:24 gw php-fpm[38147]: /rc.newwanip: IP Address has changed, killing states on former IP Address x.x.x.213.
May  4 03:06:28 gw php-fpm[38147]: /rc.newwanip: Resyncing OpenVPN instances for interface WAN.
May  4 03:06:28 gw php-fpm[38147]: /rc.newwanip: Creating rrd update script
May  4 03:06:30 gw php[68340]: [pfBlockerNG] No changes to Firewall rules, skipping Filter Reload
May  4 03:06:30 gw php[68340]: 
May  4 03:06:30 gw php-fpm[38147]: /rc.newwanip: pfSense package system has detected an IP change or dynamic WAN reconnection - x.x.x.213 ->  y.y.y.97 - Restarting packages.

We still have this issue and as a workaround (to present out Public IP as something else) we setup an Outbound NAT mapping to select the VIP which used to be used by out PPOE connection. The reason was we have lots of services we access which are locked down to IP address, so this was easier than trying to update everything else.

Recently we've found that even with this rule, our outbound IP randomly changes to one of the other VIPs from our range. If we reboot the router then the rule works again.

Viktor Gurov wrote in #note-13:

Merged:
https://github.com/pfsense/pfsense/commit/6c98abd379b9222824ba8465c38253d6bd6f5253

Looks like it's still not working for us.
Steps to reproduce the issue:

Setup a pfsense1 instance to be used as pppoe server:

PPPoE Server Configuration
Total User Count: 255
User Max Logins: 255
Server Address: 10.0.1.1
Remote Address Range: 10.65.0.0
Subnet mask: 24
Authentication type: PAP
User table: user pass

Setup a pfsense2 instance to be used as pppoe client with virtual ips on wan interface:

Interfaces/WAN (pppoe0)
IPv4 Configuration Type: PPPoE
Username: user
Password: pass

Firewall/Virtual IPs
Virtual IP Address: 10.20.30.41/30, Interface: WAN, Type: IP Alias
Virtual IP Address: 10.20.30.42/30, Interface: WAN, Type: IP Alias

Connect both pfsense2 wan interface and pppoe server interface on pfsense1 to the same virtual switch and wait for the ppp connection to be established.

Disconnect the pppoe server interface on pfsense1 and wait for the ppp connection to be terminated on pfsense2.

Connect the pppoe server interface on pfsense1 and wait for the ppp connection to be restored on pfsense2.

After the reconnection $curwanip will be 10.20.30.41 instead of 10.65.0.1

Relevant log entries:

May  9 13:33:01 pfSense ppp[36255]: [wan] IPCP: state change Ack-Sent --> Opened
May  9 13:33:01 pfSense ppp[36255]: [wan] IPCP: LayerUp
May  9 13:33:01 pfSense ppp[36255]: [wan]   10.65.0.1 -> 10.0.1.1
May  9 13:33:01 pfSense check_reload_status[395]: Rewriting resolv.conf
May  9 13:33:02 pfSense check_reload_status[395]: rc.newwanip starting pppoe0
May  9 13:33:02 pfSense ppp[36255]: [wan] IFACE: Up event
May  9 13:33:02 pfSense ppp[36255]: [wan] IFACE: Rename interface ng0 to pppoe0
May  9 13:33:02 pfSense ppp[36255]: [wan] IFACE: Add description "WAN" 
May  9 13:33:03 pfSense php-fpm[367]: /rc.newwanip: rc.newwanip: Info: starting on pppoe0.
May  9 13:33:03 pfSense php-fpm[367]: /rc.newwanip: rc.newwanip: on (IP address: 10.20.30.41) (interface: WAN[wan]) (real interface: pppoe0).
May  9 13:33:05 pfSense rc.gateway_alarm[36031]: >>> Gateway alarm: WAN_PPPOE (Addr:10.0.1.1 Alarm:1 RTT:0.000ms RTTsd:0.000ms Loss:100%)
May  9 13:33:05 pfSense php-fpm[367]: /rc.newwanip: Gateway, none 'available' for inet, use the first one configured. 'WAN_PPPOE'
May  9 13:33:05 pfSense check_reload_status[395]: updating dyndns WAN_PPPOE
May  9 13:33:05 pfSense check_reload_status[395]: Restarting IPsec tunnels
May  9 13:33:05 pfSense check_reload_status[395]: Restarting OpenVPN tunnels/interfaces
May  9 13:33:05 pfSense check_reload_status[395]: Reloading filter
May  9 13:33:05 pfSense php-fpm[367]: /rc.newwanip: Default gateway setting Interface WAN_PPPOE Gateway as default.
May  9 13:33:05 pfSense php-fpm[367]: /rc.newwanip: Gateway, none 'available' for inet6, use the first one configured. ''
May  9 13:33:05 pfSense php-fpm[367]: /rc.newwanip: IP Address has changed, killing states on former IP Address 10.65.0.0.
May  9 13:33:07 pfSense php-fpm[366]: /rc.openvpn: Gateway, none 'available' for inet, use the first one configured. 'WAN_PPPOE'
May  9 13:33:07 pfSense php-fpm[366]: /rc.openvpn: Gateway, none 'available' for inet6, use the first one configured. ''
May  9 13:34:02 pfSense php-fpm[367]: /rc.newwanip: Resyncing OpenVPN instances for interface WAN.
May  9 13:34:02 pfSense php-fpm[367]: /rc.newwanip: Creating rrd update script
May  9 13:34:04 pfSense php-fpm[367]: /rc.newwanip: pfSense package system has detected an IP change or dynamic WAN reconnection - 10.65.0.0 ->  10.20.30.41 - Restarting packages.
May  9 13:34:04 pfSense check_reload_status[395]: Starting packages
May  9 13:34:05 pfSense php-fpm[366]: /rc.start_packages: Restarting/Starting all packages.

/var/db/wan_ip

[2.6.0-RELEASE][admin@pfSense.home.arpa]/root: cat /var/db/wan_ip
10.20.30.41

fix:
https://gitlab.netgate.com/pfSense/pfSense/-/merge_requests/777

We've installed 22.05 on our Netgate 2100 appliance and it's still assigning the wrong IP address to the WAN interface and not the one shown in the PPP logs.

Dan Rice wrote in #note-23:

We've installed 22.05 on our Netgate 2100 appliance and it's still assigning the wrong IP address to the WAN interface and not the one shown in the PPP logs.

Please test again after reinstalling 22.05 now that it has been released and provide relevant logs and steps taken.