Regression #12816
closed
Namecheap Dynamic DNS responses are not parsed properly
100%
Description
Namecheap dynamic DNS updates are succeeding on the server side but the dynamic DNS code can't interpret the response. It appears that the encoding is being reported wrong in the response from Namecheap which makes it fail to parse as XML.
The response looks like this now:
<?xml version="1.0" encoding="utf-16"?>
<interface-response>
<Command>SETDNSHOST</Command>
<Language>eng</Language>
<IP>x.x.x.x</IP>
<ErrCount>0</ErrCount>
<errors />
<ResponseCount>0</ResponseCount>
<responses />
<Done>true</Done>
<debug><![CDATA[]]></debug>
</interface-response>
Note that it claims the encoding is UTF-16.
The response array is empty in the code because it is failing to parse due to that error. Trying to run the response through xmllint shows:
parser error : Document labelled UTF-16 but has UTF-8 content
If we drop the first line from the response, that function parses the XML as expected:
diff --git a/src/etc/inc/dyndns.class b/src/etc/inc/dyndns.class
index 8065879cf3..55368b162e 100644
--- a/src/etc/inc/dyndns.class
+++ b/src/etc/inc/dyndns.class
@@ -2412,6 +2412,10 @@
case 'namecheap':
$tmp = str_replace("^M", "", $data);
$ncresponse = @xml2array($tmp);
+ /* If XML parsing failed, it may be due to mismatched encoding on the response, drop the xml line. */
+ if (empty($ncresponse)) {
+ $ncresponse = @xml2array(substr($tmp, strpos($tmp, "\n") + 1));
+ }
if (preg_match("/internal server error/i", $data)) {
$status = $status_intro . $error_str . gettext("Server side error.");
} else if (preg_match("/request is badly formed/i", $data)) {
I'm not sure if that's the best solution here, though it does work around the problem.
Files
Updated by Jim Pingle almost 3 years ago
MR for the above change, but only use it if we can't come up with a better solution:
https://gitlab.netgate.com/pfSense/pfSense/-/merge_requests/627
Updated by Jim Pingle almost 3 years ago
- Assignee set to Jim Pingle
The MR should be good enough for now, I've tested it on a few more Namecheap DDNS entries on multiple systems and it allows them to properly parse the update response.
Updated by Jim Pingle almost 3 years ago
- Status changed from New to Feedback
- % Done changed from 0 to 100
Applied in changeset 4612721800a1b25bb1fb2d4d7c4ceea6f44f208e.
Updated by
Updated by Jim Pingle over 2 years ago
- Subject changed from Namecheap dynamic DNS update response is not being parsed properly to Namecheap Dynamic DNS responses are not parsed properly
Updating subject for release notes.
Updated by Jim Pingle over 2 years ago
- Status changed from Feedback to Resolved
This has been working well since it went in.
Updated by Jim Pingle over 2 years ago
- Status changed from Resolved to In Progress
- Plus Target Version changed from 22.05 to 22.11
This seems to have broken again in almost the exact same way, but the previous workaround no longer functions. The last fix relied on the XML line being a separate line so is prone to problems with differences in whitespace. I have a more general fix that seems to work on the current response in a better way plus an additional variation of the previous workaround in case that fails.
diff --git a/src/etc/inc/dyndns.class b/src/etc/inc/dyndns.class
index 63936a76ec..e71de40a50 100644
--- a/src/etc/inc/dyndns.class
+++ b/src/etc/inc/dyndns.class
@@ -2415,9 +2415,19 @@
case 'namecheap':
$tmp = str_replace("^M", "", $data);
$ncresponse = @xml2array($tmp);
- /* If XML parsing failed, it may be due to mismatched encoding on the response, drop the xml line. */
+ /* If XML parsing failed, it may be due to unsupported encoding on the response. */
if (empty($ncresponse)) {
- $ncresponse = @xml2array(substr($tmp, strpos($tmp, "\n") + 1));
+ mb_convert_encoding($tmp, 'UTF-8', 'UTF-16');
+ $tmp = str_ireplace('utf-16', 'utf-8', $tmp);
+ $ncresponse = @xml2array($tmp);
+ }
+ /* If it's still empty, try parsing without the XML definition */
+ if (empty($ncresponse)) {
+ $matches = [];
+ preg_match("/(<?xml.*?>)(.*)/", $tmp, $matches);
+ if (count($matches) == 3) {
+ $ncresponse = @xml2array($matches[2]);
+ }
}
if (preg_match("/internal server error/i", $data)) {
$status = $status_intro . $error_str . gettext("Server side error.");
Updated by Lev Prokofev over 2 years ago
- File 2022-09-10_09-36-54.png 2022-09-10_09-36-54.png added
- File 2022-09-10_09-35-54.png 2022-09-10_09-35-54.png added
Working fine for me with the latest diff
Updated by Danilo Zrenjanin over 2 years ago
Tested on:
2.7.0-DEVELOPMENT (amd64) built on Fri Sep 16 06:04:44 UTC 2022 FreeBSD 14.0-CURRENT
After clicking the Save & Force update button, I get 504 Gateway Time-out. However, it does update the IP address to Namecheap successfully.
Here are the logs:
Sep 17 10:39:05 php-fpm 58912 /services_dyndns_edit.php: phpDynDNS: updating cache file /conf/dyndns_wannamecheap'host.buyremoteit.com'1.cache: 1x.xx.xx.xx Sep 17 10:39:05 php-fpm 58912 /services_dyndns_edit.php: phpDynDNS (host): (Success) IP Address Updated Successfully! Sep 17 10:41:27 check_reload_status 390 Syncing firewall Sep 17 10:42:04 nginx 2022/09/17 10:42:04 [error] 55268#100267: *232 upstream timed out (60: Operation timed out) while reading response header from upstream, client: 192.168.20.10, server: , request: "POST /services_dyndns_edit.php?id=1 HTTP/2.0", upstream: "fastcgi://unix:/var/run/php-fpm.socket", host: "192.168.20.1", referrer: "https://192.168.20.1/services_dyndns_edit.php?id=1"
Updated by Danilo Zrenjanin over 2 years ago
Tested the latest patch on the:
22.05-RELEASE (arm) built on Wed Jun 22 18:56:40 UTC 2022 FreeBSD 12.3-STABLE
It works flawlessly.
Updated by Lev Prokofev over 2 years ago
Can confirm the behavior on 2.7.0-DEV (built on Fri Sep 16 06:04:44 UTC 2022)
Sep 17 12:40:35 php-fpm 360 /services_dyndns_edit.php: phpDynDNS: updating cache file /conf/dyndns_wannamecheap'pf2.proxxxx.xyz'0.cache: 2xx.2x.xx.xx
Sep 17 12:40:35 php-fpm 360 /services_dyndns_edit.php: phpDynDNS (pf2): (Success) IP Address Updated Successfully!
Sep 17 12:43:32 nginx 2022/09/17 12:43:32 [error] 85111#100232: *103 upstream timed out (60: Operation timed out) while reading response header from upstream, client: 172.21.102.8, server: , request: "POST /services_dyndns_edit.php?id=0 HTTP/2.0", upstream: "fastcgi://unix:/var/run/php-fpm.socket", host: "192.168.200.40", referrer: "https://192.168.200.40/services_dyndns_edit.php?id=0"
It gives 504 Gateway Time-out and doesn't forward you to https://xxx.xxx.xxx.xxx/services_dyndns_edit.php back.
Updated by Jim Pingle over 2 years ago
- Status changed from Feedback to Resolved
The GUI timeout thing is unrelated to Namecheap, might be a regression of #12870 but it's not relevant here.
As long as the entry is updated at Namecheap we can call this one solved.
Updated by Jim Pingle about 2 years ago
- Plus Target Version changed from 22.11 to 23.01