Project

General

Profile

Actions

Regression #12816

closed

Namecheap Dynamic DNS responses are not parsed properly

Added by Jim Pingle about 2 years ago. Updated over 1 year ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Dynamic DNS
Target version:
Start date:
Due date:
% Done:

100%

Estimated time:
Plus Target Version:
23.01
Release Notes:
Default
Affected Version:
Affected Architecture:

Description

Namecheap dynamic DNS updates are succeeding on the server side but the dynamic DNS code can't interpret the response. It appears that the encoding is being reported wrong in the response from Namecheap which makes it fail to parse as XML.

The response looks like this now:

<?xml version="1.0" encoding="utf-16"?>
<interface-response>
  <Command>SETDNSHOST</Command>
  <Language>eng</Language>
  <IP>x.x.x.x</IP>
  <ErrCount>0</ErrCount>
  <errors />
  <ResponseCount>0</ResponseCount>
  <responses />
  <Done>true</Done>
  <debug><![CDATA[]]></debug>
</interface-response>

Note that it claims the encoding is UTF-16.

The response array is empty in the code because it is failing to parse due to that error. Trying to run the response through xmllint shows:

parser error : Document labelled UTF-16 but has UTF-8 content

If we drop the first line from the response, that function parses the XML as expected:

diff --git a/src/etc/inc/dyndns.class b/src/etc/inc/dyndns.class
index 8065879cf3..55368b162e 100644
--- a/src/etc/inc/dyndns.class
+++ b/src/etc/inc/dyndns.class
@@ -2412,6 +2412,10 @@
                                case 'namecheap':
                                        $tmp = str_replace("^M", "", $data);
                                        $ncresponse = @xml2array($tmp);
+                                       /* If XML parsing failed, it may be due to mismatched encoding on the response, drop the xml line. */
+                                       if (empty($ncresponse)) {
+                                               $ncresponse = @xml2array(substr($tmp, strpos($tmp, "\n") + 1));
+                                       }
                                        if (preg_match("/internal server error/i", $data)) {
                                                $status = $status_intro . $error_str . gettext("Server side error.");
                                        } else if (preg_match("/request is badly formed/i", $data)) {

I'm not sure if that's the best solution here, though it does work around the problem.


Files

2022-09-10_09-36-54.png (13.2 KB) 2022-09-10_09-36-54.png Lev Prokofev, 09/10/2022 01:37 AM
2022-09-10_09-35-54.png (17.1 KB) 2022-09-10_09-35-54.png Lev Prokofev, 09/10/2022 01:37 AM
Actions #1

Updated by Jim Pingle about 2 years ago

MR for the above change, but only use it if we can't come up with a better solution:

https://gitlab.netgate.com/pfSense/pfSense/-/merge_requests/627

Actions #2

Updated by Jim Pingle about 2 years ago

  • Assignee set to Jim Pingle

The MR should be good enough for now, I've tested it on a few more Namecheap DDNS entries on multiple systems and it allows them to properly parse the update response.

Actions #3

Updated by Jim Pingle about 2 years ago

  • Status changed from New to Feedback
  • % Done changed from 0 to 100
Actions #4

Updated by Sish Kitane about 2 years ago

Fixed for me thank you

Actions #5

Updated by Jim Pingle about 2 years ago

  • Subject changed from Namecheap dynamic DNS update response is not being parsed properly to Namecheap Dynamic DNS responses are not parsed properly

Updating subject for release notes.

Actions #6

Updated by Jim Pingle about 2 years ago

  • Status changed from Feedback to Resolved

This has been working well since it went in.

Actions #7

Updated by Jim Pingle over 1 year ago

  • Status changed from Resolved to In Progress
  • Plus Target Version changed from 22.05 to 22.11

This seems to have broken again in almost the exact same way, but the previous workaround no longer functions. The last fix relied on the XML line being a separate line so is prone to problems with differences in whitespace. I have a more general fix that seems to work on the current response in a better way plus an additional variation of the previous workaround in case that fails.

diff --git a/src/etc/inc/dyndns.class b/src/etc/inc/dyndns.class
index 63936a76ec..e71de40a50 100644
--- a/src/etc/inc/dyndns.class
+++ b/src/etc/inc/dyndns.class
@@ -2415,9 +2415,19 @@
                 case 'namecheap':
                     $tmp = str_replace("^M", "", $data);
                     $ncresponse = @xml2array($tmp);
-                    /* If XML parsing failed, it may be due to mismatched encoding on the response, drop the xml line. */
+                    /* If XML parsing failed, it may be due to unsupported encoding on the response. */
                     if (empty($ncresponse)) {
-                        $ncresponse = @xml2array(substr($tmp, strpos($tmp, "\n") + 1));
+                        mb_convert_encoding($tmp, 'UTF-8', 'UTF-16');
+                        $tmp = str_ireplace('utf-16', 'utf-8', $tmp);
+                        $ncresponse = @xml2array($tmp);
+                    }
+                    /* If it's still empty, try parsing without the XML definition */
+                    if (empty($ncresponse)) {
+                        $matches = [];
+                        preg_match("/(<?xml.*?>)(.*)/", $tmp, $matches);
+                        if (count($matches) == 3) {
+                            $ncresponse = @xml2array($matches[2]);
+                        }
                     }
                     if (preg_match("/internal server error/i", $data)) {
                         $status = $status_intro . $error_str . gettext("Server side error.");
Actions #8

Updated by Jim Pingle over 1 year ago

  • Status changed from In Progress to Feedback

Fix merged

Actions #9

Updated by Lev Prokofev over 1 year ago

Working fine for me with the latest diff


Actions #10

Updated by Danilo Zrenjanin over 1 year ago

Tested on:

2.7.0-DEVELOPMENT (amd64)
built on Fri Sep 16 06:04:44 UTC 2022
FreeBSD 14.0-CURRENT

After clicking the Save & Force update button, I get 504 Gateway Time-out. However, it does update the IP address to Namecheap successfully.

Here are the logs:

Sep 17 10:39:05    php-fpm    58912    /services_dyndns_edit.php: phpDynDNS: updating cache file /conf/dyndns_wannamecheap'host.buyremoteit.com'1.cache: 1x.xx.xx.xx
Sep 17 10:39:05    php-fpm    58912    /services_dyndns_edit.php: phpDynDNS (host): (Success) IP Address Updated Successfully!
Sep 17 10:41:27    check_reload_status    390    Syncing firewall
Sep 17 10:42:04    nginx        2022/09/17 10:42:04 [error] 55268#100267: *232 upstream timed out (60: Operation timed out) while reading response header from upstream, client: 192.168.20.10, server: , request: "POST /services_dyndns_edit.php?id=1 HTTP/2.0", upstream: "fastcgi://unix:/var/run/php-fpm.socket", host: "192.168.20.1", referrer: "https://192.168.20.1/services_dyndns_edit.php?id=1" 

Actions #11

Updated by Danilo Zrenjanin over 1 year ago

Tested the latest patch on the:

22.05-RELEASE (arm)
built on Wed Jun 22 18:56:40 UTC 2022
FreeBSD 12.3-STABLE

It works flawlessly.

Actions #12

Updated by Lev Prokofev over 1 year ago

Can confirm the behavior on 2.7.0-DEV (built on Fri Sep 16 06:04:44 UTC 2022)

Sep 17 12:40:35 php-fpm 360 /services_dyndns_edit.php: phpDynDNS: updating cache file /conf/dyndns_wannamecheap'pf2.proxxxx.xyz'0.cache: 2xx.2x.xx.xx
Sep 17 12:40:35 php-fpm 360 /services_dyndns_edit.php: phpDynDNS (pf2): (Success) IP Address Updated Successfully!
Sep 17 12:43:32 nginx 2022/09/17 12:43:32 [error] 85111#100232: *103 upstream timed out (60: Operation timed out) while reading response header from upstream, client: 172.21.102.8, server: , request: "POST /services_dyndns_edit.php?id=0 HTTP/2.0", upstream: "fastcgi://unix:/var/run/php-fpm.socket", host: "192.168.200.40", referrer: "https://192.168.200.40/services_dyndns_edit.php?id=0"

It gives 504 Gateway Time-out and doesn't forward you to https://xxx.xxx.xxx.xxx/services_dyndns_edit.php back.

Actions #13

Updated by Jim Pingle over 1 year ago

  • Status changed from Feedback to Resolved

The GUI timeout thing is unrelated to Namecheap, might be a regression of #12870 but it's not relevant here.

As long as the entry is updated at Namecheap we can call this one solved.

Actions #14

Updated by Jim Pingle over 1 year ago

  • Plus Target Version changed from 22.11 to 23.01
Actions #15

Updated by Jim Pingle over 1 year ago

  • Tracker changed from Bug to Regression
Actions

Also available in: Atom PDF