Project

General

Profile

Actions

Feature #12011

closed

Disable log compression on new installations when ``/var/log`` is a ZFS dataset with compression enabled

Added by Jim Pingle 12 months ago. Updated 4 months ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
Logging
Target version:
Start date:
06/08/2021
Due date:
% Done:

0%

Estimated time:
Plus Target Version:
22.01
Release Notes:
Default

Description

The default setting for log compression is currently bzip2 for all cases, which isn't ideal for every case. If /var/log is on a ZFS dataset with compression, there isn't a compelling reason to have the logs also be compressed on rotation. ZFS is smart enough to not compress already compressed files, but it still seems inefficient and wasted effort since it's easier for users to have the log files uncompressed rather than having to read/search them using compression-related tools (e.g. bzcat, bzgrep, etc.). There are also reports from users in these cases that with large log files operations such as displaying the logs in the widget and searching are slow when using log compression.

On a fresh installation we could check if /var/log is on a ZFS dataset with compression enabled and automatically disable log compression on rotation in that case. The same should not be done post install as a user may deliberately want to set it for various reasons.

We may also want to disable log compression on specific slower hardware/disks but those cases are not as easily identified.

Actions #1

Updated by Jim Pingle 11 months ago

I thought perhaps I could have the default be assumed as 'none' with ZFS but in practice that didn't go as well as I'd hoped. For fresh installs it was fine, but if someone already had ZFS and upgraded in-place, the log compression type switched and didn't work out as well. So there are two ways to proceed:

  1. Keep going this way, but also add upgrade code to reset the logs (a bit of a POLA violation)
  2. Explicitly set the type only during the very first boot before logs are configured

The second option is probably going to end up being easier and better for users in the long run.

Actions #2

Updated by Jim Pingle 9 months ago

  • Status changed from New to Feedback

Change is in now, GUI text is coming in #12012

Needs tested a few ways:

  • Clean install with ZFS should have log compression disabled by default
  • Clean install with UFS should have log compression enabled by default and set to bzip2
  • Upgraded installations with ZFS should keep their existing setting on upgrade, but should disable compression by default after a factory reset.
Actions #3

Updated by Jim Pingle 9 months ago

  • Subject changed from Disable log compression on new installations when /var/log is on ZFS dataset with compression enabled to Disable log compression on new installations when ``/var/log`` is a ZFS dataset with compression enabled

Updating subject for release notes.

Actions #4

Updated by Jordan Greene 7 months ago

2100 on ZFS upgraded to 22.01.a.20211023.0500 shows bzip2 as log compression setting. Set to none, saved and then factory defaulted system. Upon restart system still shows bzip2 as log compression setting despite factory default.

Actions #5

Updated by Jim Pingle 7 months ago

I'm fairly certain that's because the 2100 and 1100 have compression off on /var/log by default. You can confirm this by running /sbin/zfs get -H compression /var/log. On my 1100, it's off.

This function only disables compression when using ZFS and when compression for the ZFS /var/log dataset is on.

Actions #6

Updated by Jim Pingle 7 months ago

  • Plus Target Version changed from 21.09 to 22.01
Actions #7

Updated by Jim Pingle 7 months ago

If you wipe and reload a 2100 or 1100 on a current 22.01 snapshot and use ZFS it will have lz4 compression on /var/log and would be a valid test.

Actions #8

Updated by Jordan Greene 7 months ago

clean install of 22.01.a.20211030.0500 on 1100 using ZFS, default selection under Status>Systems Logs>Settings>Log Compression shows bzip2 - /sbin/zfs get -H compression /var/log shows lz4
changing setting to none, saving and rebooting still shows lz4 when running the previous command. reverting back to bzip2 (the default gui selection following clean install) shows lz4 as well.

Actions #9

Updated by Viktor Gurov 7 months ago

Jordan Greene wrote in #note-8:

clean install of 22.01.a.20211030.0500 on 1100 using ZFS, default selection under Status>Systems Logs>Settings>Log Compression shows bzip2

fix:
https://gitlab.netgate.com/pfSense/pfSense/-/merge_requests/444

Actions #10

Updated by Jim Pingle 7 months ago

  • Status changed from Feedback to Pull Request Review
  • Assignee changed from Jim Pingle to Viktor Gurov
Actions #11

Updated by Viktor Gurov 7 months ago

  • Status changed from Pull Request Review to Feedback
Actions #12

Updated by Jim Pingle 4 months ago

  • Status changed from Feedback to Closed

Checked a few things I've recently reinstalled with zfs and they appear to have disabled compression on their own as expected with this.

Actions

Also available in: Atom PDF