Bug #5472
closed
ipsec_getpolicybyaddr() crash
0%
Description
From ticket: AUT-75761
Firewall has a large number of IPSec tunnels and is seeing unexplained crashes.
db:0:kdb.enter.default> bt
Tracing pid 12 tid 100040 td 0xfffff800036a9920
key_allocsp() at key_allocsp+0x256/frame 0xfffffe00655674d0
ipsec_getpolicybyaddr() at ipsec_getpolicybyaddr+0x8d/frame 0xfffffe0065567540
ipsec4_checkpolicy() at ipsec4_checkpolicy+0x29/frame 0xfffffe0065567560
ip_ipsec_output() at ip_ipsec_output+0x8a/frame 0xfffffe00655675a0
ip_output() at ip_output+0x966/frame 0xfffffe00655676a0
ip_forward() at ip_forward+0x347/frame 0xfffffe0065567750
ip_input() at ip_input+0x6ec/frame 0xfffffe00655677a0
netisr_dispatch_src() at netisr_dispatch_src+0x62/frame 0xfffffe0065567810
ether_demux() at ether_demux+0x149/frame 0xfffffe0065567840
ether_nh_input() at ether_nh_input+0x347/frame 0xfffffe00655678a0
netisr_dispatch_src() at netisr_dispatch_src+0x62/frame 0xfffffe0065567910
vmxnet3_rxq_eof() at vmxnet3_rxq_eof+0x4fb/frame 0xfffffe00655679a0
vmxnet3_legacy_intr() at vmxnet3_legacy_intr+0xe1/frame 0xfffffe00655679e0
intr_event_execute_handlers() at intr_event_execute_handlers+0xab/frame 0xfffffe0065567a20
ithread_loop() at ithread_loop+0x96/frame 0xfffffe0065567a70
fork_exit() at fork_exit+0x9a/frame 0xfffffe0065567ab0
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0065567ab0
--- trap 0, rip = 0, rsp = 0xfffffe0065567b70, rbp = 0 ---
db:0:kdb.enter.default> ps
Fatal trap 12: page fault while in kernel mode
cpuid = 1; apic id = 01
fault virtual address = 0xa40c050150
fault code = supervisor read data, page not present
instruction pointer = 0x20:0xffffffff80cf0b96
stack pointer = 0x28:0xfffffe0065567440
frame pointer = 0x28:0xfffffe00655674d0
code segment = base 0x0, limit 0xfffff, type 0x1b
= DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags = interrupt enabled, resume, IOPL = 0
current process = 12 (irq257: vmx1)
version.txt06000027512621323251 7614 ustarrootwheelFreeBSD 10.1-RELEASE-p24 #0 f27a67c(releng/10.1)-dirty: Wed Nov 4 16:18:37 CST 2015
root@pfs22-amd64-builder:/usr/obj.RELENG_2_2.amd64/usr/pfSensesrc/src.RELENG_2_2/sys/pfSense_SMP.10
Additionally some log entries:
ipsec4_checkpolicy: invalid policy 3
vmx0: watchdog timeout on queue 0
vmx1: watchdog timeout on queue 0
Updated by Chris Buechler over 9 years ago
- Category set to Operating System
- Affected Version changed from 2.2.5 to 2.2.x
Updated by Chris Buechler over 9 years ago
- Status changed from New to Feedback
- Assignee set to Chris Buechler
looks to be this issue:
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=204437
patch attached there committed to RELENG_2_2 tools
Updated by Chris Buechler over 9 years ago
snapshot including that patch is finished. Smoke tested fine and sent to customer who's seeing the issue.
Updated by Jim Pingle over 9 years ago
- Status changed from Feedback to Confirmed
Still broken on 2.2.6.
BTs from 2.2.6:
db:0:kdb.enter.default> bt Tracing pid 24074 tid 100246 td 0xfffff8004a6af000 key_allocsp() at key_allocsp+0x256/frame 0xfffffe011d34e660 ipsec_getpolicybysock() at ipsec_getpolicybysock+0x127/frame 0xfffffe011d34e6a0 ipsec4_checkpolicy() at ipsec4_checkpolicy+0x1f/frame 0xfffffe011d34e6c0 ip_ipsec_output() at ip_ipsec_output+0x8a/frame 0xfffffe011d34e700 ip_output() at ip_output+0x966/frame 0xfffffe011d34e800 udp_send() at udp_send+0x972/frame 0xfffffe011d34e8d0 sosend_dgram() at sosend_dgram+0x406/frame 0xfffffe011d34e950 kern_sendit() at kern_sendit+0x205/frame 0xfffffe011d34ea00 sendit() at sendit+0x129/frame 0xfffffe011d34ea50 sys_sendto() at sys_sendto+0x4d/frame 0xfffffe011d34eaa0 amd64_syscall() at amd64_syscall+0x351/frame 0xfffffe011d34ebb0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe011d34ebb0
db:0:kdb.enter.default> bt Tracing pid 51495 tid 100123 td 0xfffff8000542c000 key_delsp() at key_delsp+0x2f/frame 0xfffffe011d37b660 _key_freesp() at _key_freesp+0xa0/frame 0xfffffe011d37b6a0 ipsec4_checkpolicy() at ipsec4_checkpolicy+0xd5/frame 0xfffffe011d37b6c0 ip_ipsec_output() at ip_ipsec_output+0x8a/frame 0xfffffe011d37b700 ip_output() at ip_output+0x966/frame 0xfffffe011d37b800 udp_send() at udp_send+0x972/frame 0xfffffe011d37b8d0 sosend_dgram() at sosend_dgram+0x406/frame 0xfffffe011d37b950 kern_sendit() at kern_sendit+0x205/frame 0xfffffe011d37ba00 sendit() at sendit+0x129/frame 0xfffffe011d37ba50 sys_sendto() at sys_sendto+0x4d/frame 0xfffffe011d37baa0 amd64_syscall() at amd64_syscall+0x351/frame 0xfffffe011d37bbb0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe011d37bbb0
key_allocsp() at key_allocsp+0x256/frame 0xfffffe0118968580 ipsec_getpolicybysock() at ipsec_getpolicybysock+0x127/frame 0xfffffe01189685c0 ipsec4_in_reject() at ipsec4_in_reject+0x2a/frame 0xfffffe01189685f0 udp_append() at udp_append+0x51/frame 0xfffffe0118968670 udp_input() at udp_input+0x809/frame 0xfffffe0118968760 ip_input() at ip_input+0x118/frame 0xfffffe01189687b0 netisr_dispatch_src() at netisr_dispatch_src+0x62/frame 0xfffffe0118968820 ether_demux() at ether_demux+0x149/frame 0xfffffe0118968850 ether_nh_input() at ether_nh_input+0x347/frame 0xfffffe01189688b0 netisr_dispatch_src() at netisr_dispatch_src+0x62/frame 0xfffffe0118968920 ether_demux() at ether_demux+0xa5/frame 0xfffffe0118968950 ether_nh_input() at ether_nh_input+0x347/frame 0xfffffe01189689b0 netisr_dispatch_src() at netisr_dispatch_src+0x62/frame 0xfffffe0118968a20 em_rxeof() at em_rxeof+0x40a/frame 0xfffffe0118968ab0 em_msix_rx() at em_msix_rx+0x35/frame 0xfffffe0118968ae0 intr_event_execute_handlers() at intr_event_execute_handlers+0xab/frame 0xfffffe0118968b20 ithread_loop() at ithread_loop+0x96/frame 0xfffffe0118968b70 fork_exit() at fork_exit+0x9a/frame 0xfffffe0118968bb0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0118968bb0
Crash reports are on the server under 93.62.*
Updated by Chris Buechler about 9 years ago
- Status changed from Confirmed to Resolved
several confirmations this isn't an issue in >=2.3.