Project

General

Profile

Actions

Feature #15934

open

Feature #15650: Kea Feature Integration for parity with ISC DHCP

Kea Lease Reclamation and Affinity Options (IPv4 and IPv6)

Added by Jim Pingle about 1 year ago. Updated 2 days ago.

Status:
New
Priority:
Low
Category:
DHCP (IPv4)
Target version:
Start date:
Due date:
% Done:

0%

Estimated time:
Plus Target Version:
26.03
Release Notes:
Default

Description

The lease allocation and reclamation behavior in Kea is more aggressive than that in ISC DHCP. As a consequence, users are being surprised by unexpected IP address changes when users leases had been fairly stable under ISC DHCP.

At the moment the only option users have to influence this is to greatly increase the default lease time (and maximum in certain cases), but Kea has other options to fine-tune its behavior.

See the Kea docs at https://kea.readthedocs.io/en/kea-2.6.1/arm/lease-expiration.html#lease-reclamation

We could use some global GUI options to tweak the behavior using the options listed in https://kea.readthedocs.io/en/kea-2.6.1/arm/lease-expiration.html#lease-reclamation-configuration-parameters -- they appear to be global options, not per-subnet, but it's possible they might work both ways. We could use default values for those which are much more lenient than Kea's own defaults.

As a part of improving the lease allocation behavior, we could also use a global option to drop or ignore DHCP release packets from clients which also trigger Kea to reclaim a lease immediately:

Feel free to kick it ahead to the next release if there is not enough time or it ends up being much more complicated.

Actions #1

Updated by Dale Harron 12 months ago

Lease Affinity provides a way to have IP/MAC association persist beyond expiry but it does not survive a reboot.
https://kea.readthedocs.io/en/kea-2.6.1/arm/lease-expiration.html#lease-reclamation-configuration-parameters (see 11.4)

Setting a default affinity value would reduce the concern until a reboot at least.
To observe this, go to /etc/inc/services.inc and add 'expired-leases-processing':
$keaconf['Dhcp4'] = [
'interfaces-config' => [
'interfaces' => []
],
'lease-database' => [
'type' => 'memfile',
'persist' => true,
'name' => $kea_var_lib . '/dhcp4.leases'
],
'expired-leases-processing' => [
'reclaim-timer-wait-time' => 3,
'hold-reclaimed-time' => 86400,
'flush-reclaimed-timer-wait-time' => 5
],
'loggers' => [[

Kea is aware of this and working on a solution, re: https://gitlab.isc.org/isc-projects/kea/-/issues/3352

Actions #2

Updated by Marcos M 9 months ago

  • Target version changed from 2.8.0 to CE-Next
  • Plus Target Version changed from 25.03 to 25.07
Actions #3

Updated by Jim Pingle 9 months ago

  • Target version changed from CE-Next to 2.9.0
Actions #4

Updated by Jim Pingle 6 months ago

  • Plus Target Version changed from 25.07 to 25.11
Actions #5

Updated by Dale Harron 4 months ago

With 2.80 and Plus 25.07, I have identified that the following as an issue and should be easy to correct.

Upon a reboot, the "Show All Configured Leases" list will include all leases, including those that are expired but still protected by "Lease Affinity". Kea's default value for hold-reclaimed-time is 3600 seconds or one hour. During that time a new lease for a new MAC will get an IP that is NOT in the Configured leases list. Once the 3600 seconds expire, Kea will remove the IP from the "Show All Configured Leases" List but typically continue to allocate new leases in a numerically higher value until the pool is fully allocated. Affinity can be further modified by setting hold-reclaimed-time in expired-leases-processing as mentioned above. This is working well and behaves as expected until a reboot.

After a reboot, the "Show All Configured Leases" List looks identical but none of the expired leases are protected by "affinity" as they behave equivalent to the hold-reclaimed-time being zero seconds and any new MAC will receive the lowest numerically available IP from the pool, even if it is in the Configured Leases List. Prior to the reboot it was protected.

Any new leases issued post-reboot will return to the behavior described in the first paragraph above. Because of this drastic change in IP allocation logic, this is a problem for many devices that require the same IP and would normally have been protected by the Affinity setting be it the default 3600 seconds or a value overridden in expired-leases-processing.

As the Configured Leases information survives the reboot intact, it should be relatively simple to configure Kea to respect Affinity as it did prior to the reboot. Simply re-calculate it, all the information needed is in the Configured Leases list.

Until this is fixed, we can not use Plus 25.07 or CE 2.80 in Kea DHCP mode and ISC must remain available. pfSense should not change it's IP allocation strategy just because the server was rebooted.

Actions #6

Updated by Jim Pingle about 1 month ago

  • Plus Target Version changed from 25.11 to 26.03
Actions #7

Updated by Dale Harron 13 days ago

Further to my comment above that was made 4 months ago. Working with 25.11 Beta Nov 26, I have confirmed that although "Show All Configured Leases" lists all leases still protected by the lease affinity timeout, they are in fact not protected at all. Kea will allocate the next numerically lowest IP in the pool when a dynamic lease is requested, independent of whether it is in the Configured leases list or not after a reboot. Prior to a reboot, Kea will respect lease affinity. This confirms all the symptoms and observations above.

In an attempt to regain lease affinity protection, I ran the following command against all leases (IP/Mac/subnet-id) in the "Show All Configured Leases" list that were expired and left there (after a reboot) respecting the lease affinity setting in services.inc (default 3600 seconds).

echo '{"command":"lease4-update", "arguments" : {"hw-address": "3c:58:cc:41:b6:2a","ip-address": "192.168.50.2","subnet-id":3} }' | socat UNIX:/var/run/kea/kea4-ctrl-socket - | jq
//Note sample mac/ip pair, this has to be changed for each entry as does the subnet-id.

This results in re-establishing the lease affintiy protection and a new expiry time set but the lease is not active, it shows offline as do other lease affintiy protected leases. Kea will no longer overwrite these leases and the problem goes away because the update re-establishes the lease affinity association, all be it starting over from the current time. When Kea fixes their failure to carry over lease affinty on a reboot, this will no longer be required.

I suggest this be implemented at power up until Kea fixes the issue in https://gitlab.isc.org/isc-projects/kea/-/issues/3352 .

Actions #8

Updated by Dale Harron 6 days ago

Unfortunately, further to my comment about a lease4-update, further testing reveals that it does not actually re-instantiate lease affinity. Once the lease expires it is NOT protected by lease affinity. A lease4-add displayed the same results. It creates or updates the Lease but does not instantiate any lease affinity. Back to the drawing board!

Actions #9

Updated by Dale Harron 2 days ago

Note: All references to reboot below also apply to stop/restart of the Kea server.

Testing reveals: KEA on a reboot will carry over all leases that are either active or expired but have not yet reached the lease affinity timeout since expiry. The active leases will not be re-assigned to a new MAC simply because they are not yet expired. Once expired though, both them and the expired leases that survive the reboot respecting the lease affinity setting can be/are re-assigned immediately. Kea will typically assign the lowest lease numerically after a reboot. All new leases through a DORA sequence will be protected for the lease affinity settings duration but NO pre-boot leases will be protected.

However, Lease Affinity settings will be used by the Lease File Cleanup routines (lfc) and both old and new leases will be reclaimed based on those settings. In fact, Lease Affinity apparently only refers to this cleanup process, not the protection of a lease from reassignment.

The protection of the lease from reassignment, while showing in the "Configured Leases" list, to a new MAC will occur on all new leases that were created through a DORA sequence post reboot while in an expired state as reflected in the Status, DHCP Leases, Show all Configured Leases list. This suggests the loss of "protection" while in an expired state in the "Configured Leases List" is related to how KEA establishes it's "State" logic for assigning new leases at restart. Normally Kea will start over at the numerically lowest lease in the pool but it appears that once it has assigned two consecutively higher leases in a row through a DORA sequence, it will only assign to the next numerically higher lease from that point on, effectively protecting some of the carry over leases surviving the reboot until lfc removes them. This is because KEA will assign a matching IP/MAC pair out of sequence if it is available in the "Configured Leases List" as long as it is requested before the post reboot/restart logic overwrites it.

Any update, recreate or add of a lease through the socket will not interact with the Kea Logic that protects the lease in an expired state, only leases initiated through a DORA process will be protected. This is true all the time, not just in relationship to a reboot/restart. Use Sockets with caution if you require Lease protection for the duration it is expired but has not yet been removed by lfc.

For this reason, suggesting Lease Affinity is the issue here is incorrect, it is related to the dynamic nature of the KEA logic for assigning new leases and that ruleset/STATE in Kea is the issue, it is what is reset/lost on a reboot/restart. Once any lease has been removed by lfc, if the same MAC requests a new lease, Kea will NOT assign the old IP, as ISC DHCP did, but will follow the current STATE logic, typically assigning the next numerically higher lease available once a couple of DORA sequence leases have been assigned after a reboot/restart.

Actions

Also available in: Atom PDF