Project

General

Profile

Bug #4931

dhcpleases misses some DHCP lease changes

Added by Karl Kec over 4 years ago. Updated about 4 years ago.

Status:
Resolved
Priority:
High
Category:
DNS Resolver
Target version:
Start date:
08/12/2015
Due date:
% Done:

100%

Estimated time:
Affected Version:
All
Affected Architecture:
All

Description

Some changes to dhcp leases made by dhcpd to its database in /var/dhcpd/var/db/dhcpd.leases, are being missed by dhcpleases. When this happens unbound is not made aware of new or expired leases.

dhcpd changes the dhcpd.leases file as dhcp leases change, but once per hour it rewrites the file from scratch. When it rewrites it, the existing file is first moved out of the way and a new file created. It appears kevent in dhcpleases (watching for changes to dhcpd.leases) is then watching the moved file, which no longer is changed as dhcp leases come and go, so unbound is unaware of dhcp lease changes. After another hour, dhcpd again re-creates the dhcpd.leases file, which causes a kqueue error in dhcpleases. dhcpleases traps this and re-opens the dhcpd.leases file. Then for the next hour dhcpleases again sees lease changes, until the file is again moved and the process starts over. So it can take up to an hour for unbound to see a new dhcp lease.

As a test, I added a 5-minute timeout to the kevent in dhcpleases, and when the timeout occurs the file is closed and re-opened (see attached patch). This hack reduces delay for unbound to see a new lease to 5 minutes, but is now essentially polling for changes to the file. Five minutes works in my environment, but may not be appropriate for others.

I think this hack could work, but ideally what's needed is a better mechanism for tracking recreation of the dhcpd.leases file.

dhcpleases.patch (917 Bytes) dhcpleases.patch Karl Kec, 08/12/2015 11:39 AM

History

#1 Updated by Chris Buechler over 4 years ago

  • Subject changed from Unbound misses some dhcp lease changes to dhcpleases misses some DHCP lease changes
  • Category set to DNS Resolver
  • Status changed from New to Confirmed
  • Priority changed from Normal to High
  • Target version set to 2.2.5
  • Affected Version changed from 2.2.4 to All
  • Affected Architecture changed from amd64 to All

Thank you! That's the root of a problem I was in the midst of troubleshooting and hadn't had time to get to the bottom of yet. It also affects DNS Forwarder since it uses dhcpleases as well.

#2 Updated by Kill Bill over 4 years ago

So... as result, this causes unbound to restart every 5 minutes (Bug #4396)? I'd rather live with the original bug!!!

#3 Updated by Chris Buechler over 4 years ago

I don't think that in and of itself would trigger a restart of unbound, as it doesn't mean it picks up a hostname change. But we definitely won't do anything that restarts unbound every 5 minutes given #4396.

#4 Updated by Adam Stylinski about 4 years ago

Forgive my ignore but wouldn't these lines of code cover the situation where the file is moved?
if ((evlist.fflags & NOTE_DELETE) || (evlist.fflags & NOTE_RENAME)) {
close(leasefd);
goto reopen;
}

Unless of course the file is being copied rather than moved. In which case the solution becomes obvious (mv rather cp).

#5 Updated by Jim Thompson about 4 years ago

  • Assignee set to Renato Botelho

#6 Updated by Luiz Souza about 4 years ago

  • Assignee changed from Renato Botelho to Luiz Souza

#7 Updated by Luiz Souza about 4 years ago

  • Status changed from Confirmed to Feedback
  • % Done changed from 0 to 100

The real issue here was a small race that came from EV_ONESHOT usage. Due to async nature of kqueue when we reopen a new file descriptor too quickly the EV_ONESHOT could still be set in kqueue making dhcpleases miss all the subsequent changes on file.

Fixed in 2.3 and RELENG_2_2.

#8 Updated by Chris Buechler about 4 years ago

  • Assignee changed from Luiz Souza to Chris Buechler

Thanks Luiz. To me for testing.

#9 Updated by Chris Buechler about 4 years ago

the same original issue still seems to exist in latest 2.2.5. Obtained leases on 50 clients, all registered fine. Another 50 a few minutes later. Those all fine. Wait an hour, and do another 50. Those don't show up in dhcpleases_entries.conf until you -HUP dhcpleases (or wait some period of time that's at least > 30 minutes).

#10 Updated by Chris Buechler about 4 years ago

  • Status changed from Feedback to Resolved

works now, across many changes of the leases file and subsequent updates.

Also available in: Atom PDF