Project

General

Profile

Actions

Regression #11839

closed

Panic on 21.05/2.6.0 snapshots when memory usage is high

Added by Jim Pingle almost 3 years ago. Updated almost 3 years ago.

Status:
Closed
Priority:
Very High
Category:
Operating System
Target version:
Start date:
04/22/2021
Due date:
% Done:

100%

Estimated time:
Plus Target Version:
21.05
Release Notes:
Force Exclusion
Affected Version:
2.6.0
Affected Architecture:

Description

On several systems (hardware and VMs) running Plus 21.05 and CE 2.6.0 snapshots I am seeing panics when the systems are experiencing high memory usage. Though memory usage alone is not always sufficient to induce a panic, the lower the memory on the system the easier it appears to be to trigger the condition.

The one system I can reproduce it on most reliably is easily triggered by an apparent bug in ospf6d which causes it to eat all available RAM after an interface event (See #11838 for details). On these systems all I need to do is save/apply on an assigned VTI interface taking part in OSPF6 or stop/start IPsec (not restart), and when IPsec reconnects it panics every time.

Another way to induce a panic in a system which is in a state where it's susceptible to panic is to run tail /dev/zero from an ssh or console shell prompt. That does not reliably induce a panic every time, however, even with multiple instances run in parallel. Thus I suspect there is some other compounding factor besides memory pressure which we haven't yet identified.

Textdumps from the most easily reproducible system are attached. The panic backtraces almost, but not entirely, happen in pf, but that may just happen to be what it was busy doing at the time.


Files

textdump-7551-21.05-3.tar (73.5 KB) textdump-7551-21.05-3.tar Jim Pingle, 04/22/2021 08:45 AM
textdump-7551-21.05-4.tar (111 KB) textdump-7551-21.05-4.tar Jim Pingle, 04/22/2021 08:45 AM
textdump-7551-21.05-1.tar (73.5 KB) textdump-7551-21.05-1.tar Jim Pingle, 04/22/2021 08:45 AM
textdump-7551-21.05-0.tar (90 KB) textdump-7551-21.05-0.tar Jim Pingle, 04/22/2021 08:45 AM
textdump-7551-21.05-2.tar (95.5 KB) textdump-7551-21.05-2.tar Jim Pingle, 04/22/2021 08:45 AM
textdump-7100-21.05-0.tar (138 KB) textdump-7100-21.05-0.tar Jim Pingle, 04/22/2021 10:11 AM
textdump-ESX-2.6.0-1.tar (95 KB) textdump-ESX-2.6.0-1.tar Jim Pingle, 04/27/2021 09:27 AM
textdump-ESX-2.6.0-0.tar (77.5 KB) textdump-ESX-2.6.0-0.tar Jim Pingle, 04/27/2021 09:27 AM
textdump-ESX-2.6.0-2.tar (142 KB) textdump-ESX-2.6.0-2.tar Jim Pingle, 05/07/2021 09:38 AM
textdump-KVM-2.6.0-3.tar (101 KB) textdump-KVM-2.6.0-3.tar Jim Pingle, 05/07/2021 09:38 AM
config-pfSense.home.arpa-20210518194823.xml (21.1 KB) config-pfSense.home.arpa-20210518194823.xml Jim Pingle, 05/18/2021 02:57 PM
textdump-KVM-21.05-4.tar (142 KB) textdump-KVM-21.05-4.tar Jim Pingle, 05/19/2021 10:03 AM
textdump-KVM-21.05-3.tar (128 KB) textdump-KVM-21.05-3.tar Jim Pingle, 05/19/2021 10:03 AM
textdump-KVM-21.05-1.tar (100 KB) textdump-KVM-21.05-1.tar Jim Pingle, 05/19/2021 10:03 AM
textdump-KVM-21.05-0.tar (72 KB) textdump-KVM-21.05-0.tar Jim Pingle, 05/19/2021 10:03 AM
textdump-KVM-21.05-2.tar (114 KB) textdump-KVM-21.05-2.tar Jim Pingle, 05/19/2021 10:03 AM
textdump-KVM-2.6.0-8.tar (154 KB) textdump-KVM-2.6.0-8.tar Jim Pingle, 05/19/2021 10:03 AM
textdump-ESX-2.6.0-3.tar (154 KB) textdump-ESX-2.6.0-3.tar Jim Pingle, 05/19/2021 10:03 AM
textdump-KVM-2.6.0-6.tar (90.5 KB) textdump-KVM-2.6.0-6.tar Jim Pingle, 05/19/2021 10:03 AM
textdump-KVM-2.6.0-7.tar (126 KB) textdump-KVM-2.6.0-7.tar Jim Pingle, 05/19/2021 10:03 AM
textdump-APU-2.6.0-0.tar (84 KB) textdump-APU-2.6.0-0.tar Jim Pingle, 05/19/2021 10:03 AM
Actions

Also available in: Atom PDF