Bug #9634
closed
rc.newwanipv6 is called although dhcp6c should discard Request messages
Added by Karl Klempner almost 5 years ago. Updated over 3 years ago.
100%
Description
pfsense sends DHCPv6 Request messsages to ff02::1:2 on its WAN interface at an interval of about 7 seconds. As this is a link-local multicast destination, it also receives those messages. This starts a chain reaction: Upon recieving the message, dhcp6c_wan_script.sh calls rc.newwanipv6 which triggers filter reload etc., pushing my APU's load from 9 to >70 %, leading to severe UDP packet loss, causing DNS request to fail, and rendering VoIP calls sustainably unbearable.
RFC 8415 (DHCPv6) clearly states, "Clients MUST discard any received Request messages" (section 16.4). Following this directive, would it be a solution to ignore REQUEST reasons in dhcp6c_wan_script.sh as we already do with RENEW and INFO reasons, or would this break something else?
Current workaround is to turn off IPv6 on the WAN interface (IPv6 Configuration Type: None).
Updated by Flole Systems almost 5 years ago
The entire script is broken, even RENEW should be ignored and just REBIND should actually matter. See #9357 for a patch, if it gets modified to ignore REQUEST reasons aswell that would already solve quite some issues, someone just has to do it.
Updated by Karl Klempner over 4 years ago
I am now testing the following, modified /var/etc/dhcp6c_wan_script.sh:
#!/bin/sh
# This shell script launches /etc/rc.newwanipv6 with a interface argument.
dmips=${new_domain_name_servers}
dmnames=${new_domain_name}
case $REASON in
# Removed for testing:
# REQUEST)
# /usr/local/sbin/fcgicli -f /etc/rc.newwanipv6 -d "interface=pppoe0&dmnames=${dmnames}&dmips=${dmips}"
# ;;
REBIND)
;;
RELEASE)
/usr/local/sbin/fcgicli -f /etc/rc.newwanipv6 -d "interface=pppoe0&dmnames=${dmnames}&dmips=${dmips}"
;;
# Replaced for testing:
# RENEW|INFO)
RENEW|INFO|REQUEST)
esac
At first sight the processor load spikes seem to have gone. Let me watch this for a few days.
Updated by Karl Klempner over 4 years ago
Seems to work just fine.
(I had to disable the periodic reset of the PPPoE-(WAN-)Interface for the test to work, because anytime the interface is reset, the directory /var/etc and my modified script would be flushed.)
Updated by Viktor Gurov about 4 years ago
https://tools.ietf.org/html/rfc8415#section-16.4
ignore REQUEST messages fix:
https://github.com/pfsense/pfsense/pull/4167
Updated by Jim Pingle about 4 years ago
- Status changed from New to Pull Request Review
- Priority changed from High to Normal
- Target version set to 2.5.0
Updated by Renato Botelho almost 4 years ago
- Status changed from Pull Request Review to Feedback
- Assignee set to Renato Botelho
- % Done changed from 0 to 100
PR has been merged. Thanks!
Updated by Jim Pingle almost 4 years ago
- Target version changed from 2.5.0 to 2.4.5-p1
Updated by Jim Pingle almost 4 years ago
- Status changed from Feedback to Resolved
This appears to be working OK. The generated script matches the new code, and I no longer see any logged messages about actions being taken for REQUEST messages. Interfaces still have DHCPv6-obtained IPv6 addresses.
Updated by Daryl Morse almost 4 years ago
Jim Pingle wrote:
This appears to be working OK. The generated script matches the new code, and I no longer see any logged messages about actions being taken for REQUEST messages. Interfaces still have DHCPv6-obtained IPv6 addresses.
This patch has an unintended negative consequence. For my ISP (Telus), the setting, "Do not wait for RA" is required. The normal sequence to start the WAN interface is the following:
send dhcpv6 solicit
receive dhcpv6 advertise
send dhcpv6 request
receive dhcpv6 reply
send icmpv6 router solicitation
receive icmpv6 router advertisement
Later on, the router sends dhcpv6 renew and receives dhcpv6 reply, to renew the lease. Every 20-30 minutes or so, the gateway sends an icmpv6 router advertisement.
With the patch installed, the router never sends the icmpv6 router solicitation, so it never receives the icmpv6 router advertisement.
I would be happy to test a revised patch.
Updated by Daryl Morse almost 4 years ago
The description says, "pfsense sends DHCPv6 Request messsages to ff02::1:2 on its WAN interface at an interval of about 7 seconds." Based on closely observing ICMPv6 and DHCPv6 packets on the WAN interface for numerous versions of pfSense over the past few years, this seems to be highly unusual. I've never once seen this behaviour. The only time I observe pfSense sending a DHCPv6 Request is during the initialization of the WAN interface. After that, pfSense doesn't ever send DHCPv6 Request messages, rather it periodically sends DHCPv6 Renew messages and receives a DHCPv6 Reply from the ISP gateway. I'm not sure what caused the behaviour, but the "fix" has broken code that worked well for a long time.
Updated by Karl Klempner almost 4 years ago
The patch is to adhere to the standard: DHCPv6 "clients MUST discard any received Request messages". If fixing that really breaks something else, the work continues in a different place. I suggest you open a new bug ID.
Updated by Jim Pingle almost 4 years ago
- Status changed from Resolved to New
The intent of the patch was to not run rc.newwanipv6 and the "without RA" path wasn't doing that, it was running rtsold. But there was a debug message which said rc.newwanipv6, which was incorrect. I added back the REQUEST bits just for the "without RA" code path, which should fix your case without re-breaking the original problem.
It is concerning that the other path relies on REQUEST, it would appear your ISP is violating the RFC, but that's not exactly surprising.
Updated by Daryl Morse almost 4 years ago
Jim Pingle wrote:
The intent of the patch was to not run rc.newwanipv6 and the "without RA" path wasn't doing that, it was running rtsold. But there was a debug message which said rc.newwanipv6, which was incorrect. I added back the REQUEST bits just for the "without RA" code path, which should fix your case without re-breaking the original problem.
Thank you for your reply. If I update the snapshot, will I get the updated patch, or do I need to manually install it?
Jim Pingle wrote:
It is concerning that the other path relies on REQUEST, it would appear your ISP is violating the RFC, but that's not exactly surprising.
I'm unclear what you mean by this comment about relying on REQUEST. What's wrong with using REQUEST? I captured the packets using wireshark and I can post them if it would be of interest.
I've had numerous discussions with an engineer who works for ISP regarding their insistence on not replying to icmpv6 router solicitation before dhcpv6 solicit / advertise / request / reply. They insist this is for "security". Right... If I understand the RFC correctly, it's a violation for a router to not reply to a router solicitation within MAX_RA_DELAY_TIME, which is 0.5 seconds. Also, the other issue is that the "other configuration" flag is not set in the RA. They say this is not an issue, because the client would have already received the information in the dhcpv6 reply.
Updated by Jim Pingle almost 4 years ago
Daryl Morse wrote:
Jim Pingle wrote:
The intent of the patch was to not run rc.newwanipv6 and the "without RA" path wasn't doing that, it was running rtsold. But there was a debug message which said rc.newwanipv6, which was incorrect. I added back the REQUEST bits just for the "without RA" code path, which should fix your case without re-breaking the original problem.
Thank you for your reply. If I update the snapshot, will I get the updated patch, or do I need to manually install it?
It is already in the latest snapshot, so if you are following 2.5.0 snapshots you can update and get it.
Jim Pingle wrote:
It is concerning that the other path relies on REQUEST, it would appear your ISP is violating the RFC, but that's not exactly surprising.
I'm unclear what you mean by this comment about relying on REQUEST. What's wrong with using REQUEST? I captured the packets using wireshark and I can post them if it would be of interest.
https://tools.ietf.org/html/rfc8415#section-16.4
16.4. Request Message Clients MUST discard any received Request messages.
By taking action we aren't technically discarding the message. It shouldn't be coming to a client at all.
Updated by Daryl Morse almost 4 years ago
Jim Pingle wrote:
Daryl Morse wrote:
Jim Pingle wrote:
The intent of the patch was to not run rc.newwanipv6 and the "without RA" path wasn't doing that, it was running rtsold. But there was a debug message which said rc.newwanipv6, which was incorrect. I added back the REQUEST bits just for the "without RA" code path, which should fix your case without re-breaking the original problem.
Thank you for your reply. If I update the snapshot, will I get the updated patch, or do I need to manually install it?
It is already in the latest snapshot, so if you are following 2.5.0 snapshots you can update and get it.
I updated and it's working fine. Thank you for taking such quick action.
Jim Pingle wrote:
Daryl Morse wrote:
Jim Pingle wrote:
The intent of the patch was to not run rc.newwanipv6 and the "without RA" path wasn't doing that, it was running rtsold. But there was a debug message which said rc.newwanipv6, which was incorrect. I added back the REQUEST bits just for the "without RA" code path, which should fix your case without re-breaking the original problem.
Thank you for your reply. If I update the snapshot, will I get the updated patch, or do I need to manually install it?
It is already in the latest snapshot, so if you are following 2.5.0 snapshots you can update and get it.
Jim Pingle wrote:
It is concerning that the other path relies on REQUEST, it would appear your ISP is violating the RFC, but that's not exactly surprising.
I'm unclear what you mean by this comment about relying on REQUEST. What's wrong with using REQUEST? I captured the packets using wireshark and I can post them if it would be of interest.
https://tools.ietf.org/html/rfc8415#section-16.4
[...]
By taking action we aren't technically discarding the message. It shouldn't be coming to a client at all.
Thank you for the explanation. This doesn't apply to my situation. It's pfsense sending the request, not the ISP gateway.
Updated by Jim Pingle almost 4 years ago
Daryl Morse wrote:
Jim Pingle wrote:
By taking action we aren't technically discarding the message. It shouldn't be coming to a client at all.
Thank you for the explanation. This doesn't apply to my situation. It's pfsense sending the request, not the ISP gateway.
It doesn't matter who sent it, it should not be received by the client. The RFC doesn't say "if the server sent it", it says "Clients MUST discard any received Request messages." (period). Thus, relying on that to take action is technically in violation of the RFC.
Updated by Martin Wasley almost 4 years ago
just to put you right on this Jim as there seems some confusion. The REQUEST you see in the ENV VAR REASON is just dhcp6c saying it's received a reply to ITS request message to the server. It's not saying I have received a REQUEST message. This is an example sequence:
• Client sends a solicit from [fe80::aabb:ccff:fedd:eeff]:546 to [ff02::1:2]:547. (Client messages are sent to the multicast address, per section 14 of RFC 8415.)
• Server replies with an advertise from [fe80::0011:22ff:fe33:5566]:547 to [fe80::aabb:ccff:fedd:eeff]:546.
• Client replies with a request from [fe80::aabb:ccff:fedd:eeff]:546 to [ff02::1:2]:547.
• Server finishes with a reply from [fe80::0011:22ff:fe33:5566]:547 to [fe80::aabb:ccff:fedd:eeff]:546.
It’s when dhcp6c receives the ‘reply’ message that it runs the script and sets the env var ‘REASON’ to ‘REQUEST’ meaning it’s received a reply to ITS request message, this is when you see 'REQUEST' in the logs.