https://redmine.pfsense.org/https://redmine.pfsense.org/favicon.ico?16780521162014-03-05T01:23:53ZpfSense bugtrackerpfSense - Bug #3500: DHCP Leases List Not Showing Hostname in Some Caseshttps://redmine.pfsense.org/issues/3500?journal_id=133822014-03-05T01:23:53ZPhillip Davisphil@jankaritech.com
<ul></ul><p>Had a quick look at this. The relevant original code in /usr/local/www/status_dhcp_leases.dhcp is:<br />$splitpattern = "'BEGIN { RS=\"}\";} {for (i=1; i<=NF; i++) printf \"%s \", \$i; printf \"}\\n\";}'";</p>
<p>That splits records based on any "}" in the file. Not so good when "}" appears somewhere in the data.</p>
<p>I thought matching "}" plus "\n" would fix it:<br />$splitpattern = "'BEGIN { RS=\"}\\n\";} {for (i=1; i<=NF; i++) printf \"%s \", \$i; printf \"}\\n\";}'";</p>
<p>or looking for "\n" first:<br />$splitpattern = "'BEGIN { RS=\"\\n}\";} {for (i=1; i<=NF; i++) printf \"%s \", \$i; printf \"}\\n\";}'";</p>
<p>or looking for "\n" then "}" then "\n":<br />$splitpattern = "'BEGIN { RS=\"\\n}\\n\";} {for (i=1; i<=NF; i++) printf \"%s \", \$i; printf \"}\\n\";}'";</p>
<p>but everything I did seemed to only use the first char in RS.</p>
<p>The pfSense/FreeBSD "awk" seems to only take notice of the first char in RS. I tried to use "/usr/bin/gawk" - that should allow multiple chars in RS - but it is not anywhere to be found on pfSense distributions.</p>
<p>It could all be done in a PHP loop instead of using "awk". But that will use more CPU. If there is some other easy way to fix up the "awk" code that would be good.</p> pfSense - Bug #3500: DHCP Leases List Not Showing Hostname in Some Caseshttps://redmine.pfsense.org/issues/3500?journal_id=206412015-09-15T02:08:33ZChris Buechlercbuechler@gmail.com
<ul><li><strong>Status</strong> changed from <i>New</i> to <i>Confirmed</i></li><li><strong>Affected Version</strong> changed from <i>2.1</i> to <i>All</i></li></ul> pfSense - Bug #3500: DHCP Leases List Not Showing Hostname in Some Caseshttps://redmine.pfsense.org/issues/3500?journal_id=396752018-12-31T16:35:20ZTotio Katunio
<ul></ul><p>This bug with the } symbol in some UID fields and the missing hostname in the Web UI is still presented in version 2.4.4-RELEASE-p1<br />I created a small fix for this particular issue - PR: <a class="external" href="https://github.com/pfsense/pfsense/pull/4027">https://github.com/pfsense/pfsense/pull/4027</a></p> pfSense - Bug #3500: DHCP Leases List Not Showing Hostname in Some Caseshttps://redmine.pfsense.org/issues/3500?journal_id=396772019-01-01T08:45:56ZJoshua Sign
<ul></ul><p>Phillip Davis wrote:</p>
<blockquote>
<p>It could all be done in a PHP loop instead of using "awk".</p>
</blockquote>
<p>It should be better and easier, for sure. A simple file_get_contents(dhcpd.leases) and a loop on each lines can do the job.<br />I really don't know why using an exec...</p>
<p>@Totio Katunio :<br />According to the man page about 'uid' field : it can contains any printable chars or octal representation of non-printables chars :<br /><pre>
The client identifier is recorded as a colon-separated hexadecimal list or as a quoted string.
If it is recorded as a quoted string and it contains one or more non-printable characters,
those characters are represented as octal escapes - a backslash character followed by three octal digits.
</pre></p>
<p>So this issue is about a '}' but your solution implies usage of '#' that could also be in the uid field, isn't it ?<br />Not sure about that, but a solution that not depend about the chars in any field may be safer.</p>
<p>I just test somes commands line to get the same output as expected in normal behavior, wich is this output for my examples :<br /><pre>
cat test_leases.txt | awk '{ gsub("#.*", "");} { gsub(";", ""); print;}' | awk 'BEGIN{ RS="}";} {for (i=1; i<=NF; i++) printf "%s ", $i; printf "}\n";}'
lease 192.168.0.229 { starts 1 2011/09/1 14:01:31 ends 2 2011/09/2 14:01:31 tstp 4 2011/09/4 14:00:00 binding state active next binding state free hardware ethernet 00:0c:76:8b:c5:20 uid "\002\000\014\213\304\026" client-hostname "paul" }
lease 192.168.0.223 { starts 3 2011/08/3 12:03:29 ends 4 2011/08/4 12:03:29 tstp 6 2011/08/6 12:00:00 binding state active next binding state free hardware ethernet 00:07:E9:74:7D:78 uid "\021\010\014\213\304\026" client-hostname "jeff" }
lease 192.168.0.213 { starts 5 2011/08/5 08:03:11 ends 6 2011/09/6 09:03:11 tstp 8 2011/09/8 09:00:00 binding state active next binding state free hardware ethernet 00:07:E9:74:7D:81 uid "\001\010\014\213\104\026" client-hostname "steve" }
lease 192.168.0.224 { starts 2 2004/09/2 14:01:31 ends 3 2005/09/3 14:01:31 tstp 5 2006/09/5 23:17:32 binding state active next binding state free hardware ethernet 22:0c:56:8b:c4:01 uid "\001\010\024\213\304\026" client-hostname "tim" }
}
</pre></p>
<p>I introduce the '}' and the '#' chars in some uids.<br />And finally compare results :</p>
<p>Original code : break output by the '}' and the '#'<br /><pre>
cat test_leases.txt | awk '{ gsub("#.*", "");} { gsub(";", ""); print;}' | awk 'BEGIN{ RS="}";} {for (i=1; i<=NF; i++) printf "%s ", $i; printf "}\n";}'
lease 192.168.0.229 { starts 1 2011/09/1 14:01:31 ends 2 2011/09/2 14:01:31 tstp 4 2011/09/4 14:00:00 binding state active next binding state free hardware ethernet 00:0c:76:8b:c5:20 uid "\002\000\014\213\304\026" client-hostname "paul" }
lease 192.168.0.223 { starts 3 2011/08/3 12:03:29 ends 4 2011/08/4 12:03:29 tstp 6 2011/08/6 12:00:00 binding state active next binding state free hardware ethernet 00:07:E9:74:7D:78 uid "\021\010\014\213\304\026 }
" client-hostname "jeff" }
lease 192.168.0.213 { starts 5 2011/08/5 08:03:11 ends 6 2011/09/6 09:03:11 tstp 8 2011/09/8 09:00:00 binding state active next binding state free hardware ethernet 00:07:E9:74:7D:81 uid "\001\010\014 client-hostname "steve" }
lease 192.168.0.224 { starts 2 2004/09/2 14:01:31 ends 3 2005/09/3 14:01:31 tstp 5 2006/09/5 23:17:32 binding state active next binding state free hardware ethernet 22:0c:56:8b:c4:01 uid "\001\010\024\213\304\026" client-hostname "tim" }
</pre></p>
<p>Your workaround : breaked by '#' char only<br /><pre>
cat test_leases.txt | awk '{ gsub("#.*", "");} { gsub(";", ""); print;}' | sed 's/^}/#/g' | awk 'BEGIN{ RS="#";} {for (i=1; i<=NF; i++) printf "%s ", $i; printf "}\n";}'
lease 192.168.0.229 { starts 1 2011/09/1 14:01:31 ends 2 2011/09/2 14:01:31 tstp 4 2011/09/4 14:00:00 binding state active next binding state free hardware ethernet 00:0c:76:8b:c5:20 uid "\002\000\014\213\304\026" client-hostname "paul" }
lease 192.168.0.223 { starts 3 2011/08/3 12:03:29 ends 4 2011/08/4 12:03:29 tstp 6 2011/08/6 12:00:00 binding state active next binding state free hardware ethernet 00:07:E9:74:7D:78 uid "\021\010\014\213\304\026}" client-hostname "jeff" }
lease 192.168.0.213 { starts 5 2011/08/5 08:03:11 ends 6 2011/09/6 09:03:11 tstp 8 2011/09/8 09:00:00 binding state active next binding state free hardware ethernet 00:07:E9:74:7D:81 uid "\001\010\014 client-hostname "steve" }
lease 192.168.0.224 { starts 2 2004/09/2 14:01:31 ends 3 2005/09/3 14:01:31 tstp 5 2006/09/5 23:17:32 binding state active next binding state free hardware ethernet 22:0c:56:8b:c4:01 uid "\001\010\024\213\304\026" client-hostname "tim" }
</pre></p>
<p>A better workaround should be this command line, wich output seems to be as expected :<br /><pre>
cat test_leases.txt | sed '/^#/ d' | sed '/^$/ d' | perl -ne 's/^(..+)\n$/$1 /g; print;' | tr ';' ' ' | perl -ne 's/\s\s+/ /g; print;' ; echo "}" ;
lease 192.168.0.229 { starts 1 2011/09/1 14:01:31 ends 2 2011/09/2 14:01:31 tstp 4 2011/09/4 14:00:00 binding state active next binding state free hardware ethernet 00:0c:76:8b:c5:20 uid "\002\000\014\213\304\026" client-hostname "paul" }
lease 192.168.0.223 { starts 3 2011/08/3 12:03:29 ends 4 2011/08/4 12:03:29 tstp 6 2011/08/6 12:00:00 binding state active next binding state free hardware ethernet 00:07:E9:74:7D:78 uid "\021\010\014\213\304\026}" client-hostname "jeff" }
lease 192.168.0.213 { starts 5 2011/08/5 08:03:11 ends 6 2011/09/6 09:03:11 tstp 8 2011/09/8 09:00:00 binding state active next binding state free hardware ethernet 00:07:E9:74:7D:81 uid "\001\010\014#\213\104\026" client-hostname "steve" }
lease 192.168.0.224 { starts 2 2004/09/2 14:01:31 ends 3 2005/09/3 14:01:31 tstp 5 2006/09/5 23:17:32 binding state active next binding state free hardware ethernet 22:0c:56:8b:c4:01 uid "\001\010\024\213\304\026" client-hostname "tim" }
</pre></p>
<p>I use perl because i can't achieve this with awk regexp using crlf.<br />Unfortunately, i don't use dhcp over my pfsense actually, and i can't really test it, but the line 120 of the file '/usr/local/www/status_dhcp_leases.php' can be replaced by something like :</p>
<pre>
exec("/bin/cat {$leasesfile} | /usr/bin/sed '/^#/ d' | /usr/bin/sed '/^\$/ d' | /usr/local/bin/perl -ne 's/^(..+)\\n\$/\$1 /g; print;' | /usr/bin/tr ';' ' ' | /usr/local/bin/perl -ne 's/\\s\\s+/ /g; print;' ; echo \"}\";", $leases_content);
</pre>
<p>Corresponding in my command line above.</p>
<p>Regards<br />Josh_</p> pfSense - Bug #3500: DHCP Leases List Not Showing Hostname in Some Caseshttps://redmine.pfsense.org/issues/3500?journal_id=396792019-01-02T12:06:03ZTotio Katunio
<ul></ul><p>Well, I agree, but at least my proposal doesn't break the overall logic, the current code base already strips the # symbols, before I reuse it for the lease blocks, so it is not a regression, but an improvement (still not a complete one). This was the fastest/simplest way to fix it, without using too much more magic :)</p>
<p>Your solution is better, but I guess that we should test it directly on the pfSense host, since the tools there are pretty outdated (awk is a 2012 build binary).</p>
<p>PS: Also introducing perl, makes the pipeline execution slower but I tested it and it works ok. Maybe it will be a bit noticeable with more leases.<br />PS2: The usage of tr is also unsafe in the same regard as the # cleanup (';' is also a possible char for the uid)</p> pfSense - Bug #3500: DHCP Leases List Not Showing Hostname in Some Caseshttps://redmine.pfsense.org/issues/3500?journal_id=396822019-01-03T05:11:37ZJoshua Sign
<ul></ul><p>Hi Totio,</p>
<p>As i didn't find a way to correctly handle '\r' or '\n' with awk or sed, i switch to perl which can play with complex regexp, <br />even if its a little bit slower (need to be tested in real conditions), at least the results are corrects.</p>
<p>You are right to point the problem about tr usage and the possible usage of ';' char in uid field : i just missed it.</p>
<p>Here is a new version of the command line : 'tr' part removed and taking account that only ';' followed by '\n' will be deleted :</p>
<pre>
cat test_leases.txt | sed '/^#/ d' | sed '/^$/ d' | perl -ne 's/^(..+)(({)|;)+\n$/$1$3 /g; print;' | perl -ne 's/\s\s+/ /g; print;' ; echo "}" :
</pre>
<p>About the time to process, i put 4780 entries in my test_leases.txt :<br />- perl : about 0.218 seconds<br />- awk : about 0.124 seconds</p>
<p><ins>So you are right : using perl is slower than awk.</ins></p>
<p>But is it really significant for common usages ?<br />I supposed that many people don't have so much dhcp leases entries, but most probably 20 times less.</p> pfSense - Bug #3500: DHCP Leases List Not Showing Hostname in Some Caseshttps://redmine.pfsense.org/issues/3500?journal_id=396942019-01-04T08:44:30ZTotio Katunio
<ul></ul><p>Hi Joshua,<br />ok, so do you want me to close the current PR. I guess that you can make new PR with your implementation?</p> pfSense - Bug #3500: DHCP Leases List Not Showing Hostname in Some Caseshttps://redmine.pfsense.org/issues/3500?journal_id=396952019-01-04T10:17:45ZJoshua Sign
<ul></ul><p>I am not able to test this because i dont use this service.<br />I was just trying to help about this subject to find the best way to avoid furthers errors.</p>
<p>I think you should keep the PR and wait opinions of managers like Jim.<br />They will give you their advice and you'll be able to adapt the PR.</p>
<p>Regards</p> pfSense - Bug #3500: DHCP Leases List Not Showing Hostname in Some Caseshttps://redmine.pfsense.org/issues/3500?journal_id=408402019-06-04T08:31:14ZRenato Botelhorenato@netgate.com
<ul><li><strong>Status</strong> changed from <i>Confirmed</i> to <i>Feedback</i></li><li><strong>Assignee</strong> set to <i>Renato Botelho</i></li><li><strong>Target version</strong> set to <i>2.5.0</i></li><li><strong>% Done</strong> changed from <i>0</i> to <i>100</i></li></ul><p>I've reimplemented the function that parses dhcpd.leases</p> pfSense - Bug #3500: DHCP Leases List Not Showing Hostname in Some Caseshttps://redmine.pfsense.org/issues/3500?journal_id=408772019-06-10T07:36:29ZJim Pingle
<ul><li><strong>Status</strong> changed from <i>Feedback</i> to <i>Assigned</i></li></ul><p>Looks OK for the most part, though I do have one weird device that doesn't match in the leases database vs what is displayed in the GUI.</p>
<p>GUI shows the hostname as "Nintendo.<my domain>" but the lease DB has:</p>
<pre>
lease x.x.x.x {
[...]
client-hostname "Nintendo 3DS";
}
</pre>
<p>Since a space isn't valid in a hostname, I'm not sure what the best course of action is here. Using only the first word seems like it could be problematic. Perhaps replace spaces with dashes or remove them.</p> pfSense - Bug #3500: DHCP Leases List Not Showing Hostname in Some Caseshttps://redmine.pfsense.org/issues/3500?journal_id=419862019-08-21T10:38:51ZJim Pingle
<ul><li><strong>Category</strong> changed from <i>Web Interface</i> to <i>DHCP (IPv4)</i></li></ul> pfSense - Bug #3500: DHCP Leases List Not Showing Hostname in Some Caseshttps://redmine.pfsense.org/issues/3500?journal_id=423242019-09-12T17:46:57ZRenato Botelhorenato@netgate.com
<ul><li><strong>File</strong> <a href="/attachments/2809">Screen Shot 2019-09-12 at 19.45.13.png</a> <a class="icon-only icon-download" title="Download" href="/attachments/download/2809/Screen%20Shot%202019-09-12%20at%2019.45.13.png">Screen Shot 2019-09-12 at 19.45.13.png</a> added</li><li><strong>Status</strong> changed from <i>Assigned</i> to <i>Feedback</i></li></ul><p>Jim Pingle wrote:</p>
<blockquote>
<p>Looks OK for the most part, though I do have one weird device that doesn't match in the leases database vs what is displayed in the GUI.</p>
<p>GUI shows the hostname as "Nintendo.<my domain>" but the lease DB has:</p>
<p>[...]</p>
<p>Since a space isn't valid in a hostname, I'm not sure what the best course of action is here. Using only the first word seems like it could be problematic. Perhaps replace spaces with dashes or remove them.</p>
</blockquote>
<p>Can you test it again? I added a space on an entry here and it worked as expected. Take a look at the screenshot from status_dhcp_releases.php</p> pfSense - Bug #3500: DHCP Leases List Not Showing Hostname in Some Caseshttps://redmine.pfsense.org/issues/3500?journal_id=423292019-09-13T07:51:19ZJim Pingle
<ul><li><strong>Status</strong> changed from <i>Feedback</i> to <i>In Progress</i></li></ul><p>I'm not seeing any change here:</p>
<p>Lease DB:<br /><pre>
lease xx.xx.32.103 {
starts 5 2019/09/13 12:40:41;
ends 5 2019/09/13 14:40:41;
cltt 5 2019/09/13 12:40:41;
binding state active;
next binding state free;
rewind binding state free;
hardware ethernet 40:d2:8a:xx:xx:xx;
uid "\001@\322\212\261\223\271";
client-hostname "Nintendo 3DS";
}
lease xx.xx.32.103 {
starts 5 2019/09/13 12:40:41;
ends 5 2019/09/13 12:40:55;
tstp 5 2019/09/13 12:40:55;
cltt 5 2019/09/13 12:40:41;
binding state free;
hardware ethernet 40:d2:8a:xx:xx:xx;
uid "\001@\322\212\261\223\271";
}
</pre></p>
<p>GUI:<br /><pre>
xx.xx.32.103 40:d2:8a:xx:xx:xx (Nintendo) Nintendo.xxxxxx.org 2019/09/13 12:40:41 2019/09/13 12:40:55 online expired
</pre></p>
<p>dhcpleases:<br /><pre>
: grep -i nintendo /var/unbound/dhcpleases_entries.conf
local-data: "Nintendo.xxxxxx.org IN A xx.xx.32.103"
local-data-ptr: "xx.xx.32.103 Nintendo.xxxxxx.org"
</pre></p> pfSense - Bug #3500: DHCP Leases List Not Showing Hostname in Some Caseshttps://redmine.pfsense.org/issues/3500?journal_id=423302019-09-13T09:36:46ZJim Pingle
<ul></ul><p>It looks like dhcpleases having the wrong name is the problem here. The page is only displaying the result it received from <code>gethostbyaddr()</code> around line 697 since the last entry for that lease had no hostname. And from what I posted above, it looks like dhcpleases put in only the first word of the hostname.</p>
<p>If I bring the problem device online it shows in the list with the space in the name. So this is probably actually solved, unless you want to handle the dhcpleases part here. Or we can split that off into a new issue.</p> pfSense - Bug #3500: DHCP Leases List Not Showing Hostname in Some Caseshttps://redmine.pfsense.org/issues/3500?journal_id=423312019-09-13T09:39:31ZRenato Botelhorenato@netgate.com
<ul></ul><p>Jim Pingle wrote:</p>
<blockquote>
<p>It looks like dhcpleases having the wrong name is the problem here. The page is only displaying the result it received from <code>gethostbyaddr()</code> around line 697 since the last entry for that lease had no hostname. And from what I posted above, it looks like dhcpleases put in only the first word of the hostname.</p>
<p>If I bring the problem device online it shows in the list with the space in the name. So this is probably actually solved, unless you want to handle the dhcpleases part here. Or we can split that off into a new issue.</p>
</blockquote>
<p>Another ticket is better in my opinion. The main reported issue here is fixed.</p> pfSense - Bug #3500: DHCP Leases List Not Showing Hostname in Some Caseshttps://redmine.pfsense.org/issues/3500?journal_id=423322019-09-13T09:40:20ZJim Pingle
<ul><li><strong>Status</strong> changed from <i>In Progress</i> to <i>Resolved</i></li></ul><p>OK, I'll make one shortly. Closing this.</p> pfSense - Bug #3500: DHCP Leases List Not Showing Hostname in Some Caseshttps://redmine.pfsense.org/issues/3500?journal_id=423332019-09-13T09:44:04ZJim Pingle
<ul></ul><p>dhcpleases issue moved over to <a class="issue tracker-1 status-3 priority-4 priority-default closed" title="Bug: dhcpleases does not handle spaces in DHCP lease hostnames (Resolved)" href="https://redmine.pfsense.org/issues/9758">#9758</a></p>