pfSense

I've posted the same (?) conclusion in the forum : FreeRadius and quotas, doesn't work since 22.05

I would also like to point out that this issue is not solely related to FreeRadius Accounting Packets but also affects all VLan based Captive Portals that have Traffic Quotas. Services, Captive Portal, click on VLan Portal,

Traffic quota (Megabytes)
100 (for example)
Clients will be disconnected after exceeding this amount of traffic, inclusive of both downloads and uploads. They may log in again immediately, though. Leave this field blank for no traffic quota.

This quota also stays at Zero and is not enforced. Bandwidth and time related constraints are still working though.

Please test the attached patch with the System Patches package on pfSense+ 22.05.

Counters still zero

print_r(pfSense_pf_cp_get_eth_rule_counters("cpzoneid_2_auth/172.25.235.130_32"));
Array
(
    [0] => Array
        (
            [0] => 1
            [1] => 0
            [2] => 0
        )

    [1] => Array
        (
            [0] => 2
            [1] => 0
            [2] => 0
        )

)

pfctl -vvse -a cpzoneid_2_auth/172.25.235.130_32
@0 ether pass in quick proto 0x0800 from a8:20:66:2b:f0:d2 l3 from 172.25.235.130 to any tag cpzoneid_2_auth dnpipe 2002
  [ Evaluations: 19232     Packets: 4298      Bytes: 477032    ]
  [ Last Active Time: Sun Dec 11 21:53:56 2022 ]
@1 ether pass out quick proto 0x0800 to a8:20:66:2b:f0:d2 l3 from any to 172.25.235.130 tag cpzoneid_2_auth dnpipe 2003
  [ Evaluations: 14914     Packets: 13515     Bytes: 18008092  ]
  [ Last Active Time: Sun Dec 11 21:53:56 2022 ]

PF_IN/PF_OUT direction was mismatched with the array index into the counters that we sampled. This should be fixed in the next build.

Some success, data is now being passed to freeRadius but: (steady state stream of 33 MB/minute, single login per user, Dec 30,0600 beta release)

1. If stop/start freeRadius is set for accounting in the captive portal with or without Reauthenticate con users every minute, the data value increments at 4-9 X the true rate with the variable appearing to be time. i.e. it starts out at 1, then 2, then 4 and around 10 minutes it has logged 9X the actual data throughput. It is as if the interim accounting interval was still present and being cumulated rather than incremented as the temporary used-octets-user-mac file is not used with all data accumulated in the single used-octets-user file in var/log/radacct/datacounter/forever (or day, etc as applicable).

2. if interim is selected for accounting in the captive portal, with or without Reauthenticate con users every minute, the data value cumulates in the temporary, per connected user, used-octets-user-mac file very closely matches the actual data used (freeRadius interim intervals of 61 and default 600 secs tested). As always, the used-octets-user file is not updated until after the user logs out. With multiple logins per user, this increases overall accuracy as well but only when everyone has logged out. I have not tested this release as to if the captive portal with freeRadius authentication will correctly log all users out when the data limit is reached but in the past it did NOT. That is why stop/start freeRadius is preferred over interim in out case.

3. Reauthenticate con users every minute affected sys log contents but did not affect the above results.

4. I have not checked the captive portal data limit based on local user database authentication yet. 22.05 was not consistently enforcing it.

I am not sure if you intended stop/start freeRadius to accurately log data quotas (as in multiple logins per user), but it sure makes implementation easier if it does. pfSense correctly logs a simultaneous per-user-acct out when the single used-octets-user file maintained under stop/start freeRadius contains the total of all users data consumption on an ongoing basis to the nearest minute. I have been using that fact to insert an over-quota value into used-octets-user when we estimated that the data quota had been exceeded (using darkstat data) while redmine 13418 remains unresolved. The above over reporting of data consumption will thus log them out prematurely.

pfSense does not appear to be able to consolidate multiple (even just one) used-octets-user-mac files into the single used-octets-user file to determine data consumption, instead when it checks the single used-octets-user file for Data Quota compliance so unless the used-octets-user-mac files associated users log out, pfSense will not enforce a data Quota on the captive portal interim setting because the data use is cumulating in the user specific used-octets-user-mac file until logout when it's contents is summed with the current contents of the single used-octets-user file that pfSense checks as part of it's freeRadius support for accounting data. This is not new but in the absence of an accurate stop/start freeRadius Data log, a Data Quota can't be enforced in this setting util the cumulative data usage of logged out users exceeds the quota. Users that are not logged out are not included in the evaluation.

More extended testing demonstrates a NEW issue (see #2 point above for as tested configuration): pre-mature captive portal logout of a freeRadius authenticated user on a user account where simultaneous logins are permitted (likely also true for single login free Radius authenticated users but not tested).

With a test case of 2 simultaneous device logins to a freeRadius user account, monitoring of data logins in the device specific used-octets-user-mac files, data is being logged/incremented roughly every minute with reasonably accurate quantities in each of the 2 device specific interim freeRadius files. The used-octets-user file which summarizes the usage against that user over time (i.e. the sum of all devices/user that have been logged out already) accurately reflects prior usage. At approximately 30 minutes after around 1.5 Gb of data throughput, the Captive Portal logs the device out and merges its session data usage from the device specific used-octets-user-mac file into the used-octets-user file, deletes the used-octets-user-mac file and enters "QUOTA EXCEEDED: USER" into the System Logs/Authentication/Captive Portal Auth log. Normally that device and all other device specific sessions associated with the user account would be logged out and the User (forever) would not be able to log in again, instead receiving a "Quota Exceeded" error message. In this case though, the Device can log back in and start a new session. All other active device/user sessions continue without interruption. In this test, the second device was likewise logged out a minute or so later after similar conditions were met.

This appears to indicate an issue with the Captive Portal incorrectly implementing the freeRadius Quota, possibly because of the exponential data calc inaccuracies over time symptoms in item 1 above. Depending upon throughput, a 100 GB data quota takes approximately 1.5 GB of actual data throughput over time, typically 10 minutes to 2 hours in my testing to date.

With Captive portal set to interim and no reauthentication/minute by the captive portal, freeRadius does appear to accurately reflect the data usage to date of all "logged out devices" per user account. freeRadius also retains interim data files for devices that have not yet logged out and Captive Portal shows the device logged in for the period of time until a logout is forced by the Captive Portal due to the Captive Portal inactivity setting.

The solution that was applied for stop/start freeRadius that sends only incremental data use in each stop/start packet to freeRadius works well because it effectively bypasses the concern under 13843 as the freeRadius supplied value for max-octets-user is unimportant.

If you applied this logic to all incremental communication with freeRadius under reauthenticate every minute and interim, they too would not care about the 32 bit unsigned integer concern in 13843 and you could remove the UNUSABLE 4095 GB maximum now enforced in the freeRadius GUI front end. freeRadius will cumulate the incremental amounts as designed into user specific used-octets-user-uniqueID files and cumulate them to the used-octets-user file used by freeRadius to ascertain if a false (logout) should be sent back to pfSense Captive Portal to log the user out when the limit is reached.

If the fact multi users are logged in and freeRadius does not account for their data usage until they formally log out, or time out, is a problem to the pfSense installation, that is when either reauthenticate every minute or stop/start freeRadius are applicable and either will effect cumulation of data into the used-octets-user file so that freeRadius is tracking data usage for all users. This is by design and need not be compensated for beyond these two options.

If you need a new redmine, please raise one based on this observation but in my opinion, this is an issue related to this original request and should be done under it.

The frequency of which all interim used-octets-user-uniqueID files are consolidated into the used-octets-user file should be manageable as once a minute is too frequent and a substantial overhead for some systems with many users. I suggest you expose the frequency of both reauthenticate every minute and stop/start freeRadius to the GUI so that loading can be compenstated for by increasing the delay when deemed necessary. I do see why you might consider this a new redmine issue but I suspect is is simple to implement and it is desparately needed right now in the 23.01 release.

In way of clarification, the used-octets-user or used-octets-user-uniqueID files are currently correctly updated with incremental data once a minute (should use the freeRadius incremental update value though; 600 seconds default). The problem that is related to 13843's 32 bit unsigned integer limitation in representing the max-octets-user value is that pfSense's support for freeRadius does not let freeRadius manage the summing of the incremental consumption values into the used-octets-user file and then compare it to max-octets-user on the freeRadius side. Instead, pfSense maintains its own total and compares it to the max-octets-user value sent in the packet, yes, the value that is wrong when above 4GB due to the 32 bit unsigned integer limitations. pfSense should simply not check that value, it is not it's job to manage this data quota, that is the job of freeRadius. Thus, freeRadius is not limited by the 4GB 32 bit issue in 13843, pfSense is and only because it is not minding it's own business and instead arbitrarily disconnecting the user before receiving a request to do so from freeRadius in the interim update packet.

I will cross post this observation to 13843.