Project

General

Profile

Actions
Copy link

Bug #7925

closed

VT race condition panic at boot on ESXi 6.5.0U1 and FreeBSD 11.1 base

Added by Jim Pingle about 7 years ago. Updated about 7 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
Luiz Souza
Category:
Operating System
Target version:
2.4.1
Start date:
10/11/2017
Due date:
% Done:

100%

Estimated time:
Plus Target Version:
Release Notes:
Affected Version:
2.4.x
Affected Architecture:
amd64

Description

Some users occasionally encounter a panic during OS hardware detection on 2.4 running under ESXi 6.5.0 U1 (Build 6765664) -- before handoff to our code -- in vga_bitblt_text(). Because it is before the handoff to our code, DDB is not yet configured so the VM drops to a db> prompt and waits for input. The crash is unusual in that it does not happen to every VM at every boot. It is random and only affects a small number of reboot attempts. The crash happens before disks are mounted so filesystem corruption is not a concern.

This appears to be a confirmed FreeBSD issue:
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=217282 (has a patch availble, and it's in -CURRENT)
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=220923 (appears to be a duplicate of 217282)

From reading those bug reports, it appears to be a race condition in the VT code.

A possible workaround is to set debug.debugger_on_panic=0 in /boot/loader.conf.local and then configure a tunable in the pfSense GUI to set debug.debugger_on_panic=1 so that unrelated crash dumps can be collected afterward. That will not stop the panic, but it will allow the VM to reboot itself until it succeeds.

If the crash is in VT, another possible solution would be to switch affected VMs to the sc console by setting kern.vty=sc in /boot/loader.conf.local

So far over 12 VMs on 2.4.x and FreeBSD 11 I have only managed to make it happen once on my lab ESXi host. See the attached image for the backtrace.

Files

Download all files
Selection_704.png (15.6 KB) Selection_704.png Jim Pingle, 10/11/2017 09:21 AM
pfsense_panic.png (332 KB) pfsense_panic.png Gianluca Toso, 10/12/2017 05:47 PM
vm_bug.png (86.9 KB) vm_bug.png Constantine Kormashev, 10/18/2017 03:37 AM
vm_bug2.png (122 KB) vm_bug2.png Constantine Kormashev, 10/18/2017 03:37 AM
Selection_709.png (14.6 KB) Selection_709.png Jim Pingle, 10/18/2017 09:23 AM
Actions
Copy link

Also available in: Atom PDF