各位大佬们好,
我是菜鸡一枚,
生产的es-服务器宕机:

以下是运维提供的hs_err_pid 文件的一部分,限制长度了,只能传这么多,请各位大佬们分析以下,谢谢!

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x0000000a00000000, pid=21707, tid=0x00007fd3245f2700
#
# JRE version: Java(TM) SE Runtime Environment (8.0_151-b12) (build 1.8.0_151-b12)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.151-b12 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# C  0x0000000a00000000
#
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
# If you would like to submit a bug report, please visit:
#   http://bugreport.java.com/bugreport/crash.jsp
#

---------------  T H R E A D  ---------------

Current thread (0x00007fd2dc014800):  JavaThread "elasticsearch[es-node-3-43][search][T#17]" daemon [_thread_in_Java, id=23341, stack(0x00007fd3244f2000,0x00007fd3245f3000)]

siginfo: si_signo: 11 (SIGSEGV), si_code: 1 (SEGV_MAPERR), si_addr: 0x0000000a00000000

Registers:
RAX=0x0000000a00000000, RBX=0x00007fd466a2aa34, RCX=0x0000000a00000000, RDX=0x00007fd466a2a1a4
RSP=0x00007fd3245f0d60, RBP=0x00000002800dad08, RSI=0x00007fd466a2a1a4, RDI=0x00007fd3245f0c90
R8 =0x0000000000000006, R9 =0x00007fd465000000, R10=0x0000000000000000, R11=0x00007fd4792c5610
R12=0x0000000000000000, R13=0x00007fd20e668c63, R14=0x00000002820578a0, R15=0x00007fd2dc014800
RIP=0x0000000a00000000, EFLAGS=0x0000000000010246, CSGSFS=0x0000000000000033, ERR=0x0000000000000014
  TRAPNO=0x000000000000000e

Top of Stack: (sp=0x00007fd3245f0d60)
0x00007fd3245f0d60:   00000002807d81d0 00000007c0d65438
0x00007fd3245f0d70:   0000000000000004 00007fd4740192b0
0x00007fd3245f0d80:   00007fd3245f0dd0 00007fd4792cb6f5
0x00007fd3245f0d90:   00007fd3245f0dd0 00007fd47948d27f
0x00007fd3245f0da0:   00000007c001c6f8 00000007c0d65438
0x00007fd3245f0db0:   00007fd2dc014800 00007fd479e26a70
0x00007fd3245f0dc0:   00007fd2dc014800 0000000000000000
0x00007fd3245f0dd0:   00007fd3245f0e20 00000002804c1e80
0x00007fd3245f0de0:   00000002c39c6d70 00000002807d8230
0x00007fd3245f0df0:   00000000dc014800 00007fd4119024b0
0x00007fd3245f0e00:   000000028104aed8 0000000000000000
0x00007fd3245f0e10:   00000000500db7f6 00000002c3ce7e18
0x00007fd3245f0e20:   00000002804c1e80 00007fd46506c145
0x00007fd3245f0e30:   00000002800dad08 00007fd4673f9744
0x00007fd3245f0e40:   00000003500db80c 0000000200000002
0x00007fd3245f0e50:   00000007c0d65438 0000000000000017
0x00007fd3245f0e60:   00000002804c1e80 00000002803cd120
0x00007fd3245f0e70:   00000002806dbe70 00000002806dbe88
0x00007fd3245f0e80:   00000002800dad08 00007fd4678cb4bc
0x00007fd3245f0e90:   00000002ce414f18 00007fd46657e570
0x00007fd3245f0ea0:   00000002806dbf38 000000066c6fb590
0x00007fd3245f0eb0:   00000002ce414f18 00000002806dbf50
0x00007fd3245f0ec0:   00007fd46a010f18 00000002fa7cfa98
0x00007fd3245f0ed0:   00000000cd898016 00007fd469064cd4
0x00007fd3245f0ee0:   00000000665cf58d 00007fd46506c4db
0x00007fd3245f0ef0:   00000002804c40d8 00007fd46a00f6f0
0x00007fd3245f0f00:   00000002804c40d8 0000000000000000
0x00007fd3245f0f10:   0000000000000000 00000002803cd120
0x00007fd3245f0f20:   00000002804c40d8 00000002804c4118
0x00007fd3245f0f30:   714feefaf8014fdc 00000007c001c2e8
0x00007fd3245f0f40:   00000002806dbf60 00000002804c1e80
0x00007fd3245f0f50:   00000002800dada8 0000000333581c28 

Instructions: (pc=0x0000000a00000000)
0x00000009ffffffe0:   
[error occurred during error reporting (printing registers, top of stack, instructions near pc), id 0xb]

Register to memory mapping:

RAX=0x0000000a00000000 is an unknown value
RBX=0x00007fd466a2aa34 is at entry_point+4852 in (nmethod*)0x00007fd466a29450
RCX=0x0000000a00000000 is an unknown value
RDX=0x00007fd466a2a1a4 is at entry_point+2660 in (nmethod*)0x00007fd466a29450
RSP=0x00007fd3245f0d60 is pointing into the stack for thread: 0x00007fd2dc014800
RBP=
[error occurred during error reporting (printing register info), id 0xb]

Stack: [0x00007fd3244f2000,0x00007fd3245f3000],  sp=0x00007fd3245f0d60,  free space=1019k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
C  0x0000000a00000000


---------------  P R O C E S S  ---------------

Java Threads: ( => current thread )
  0x00007fd390024800 JavaThread "elasticsearch[es-node-3-43][flush][T#64]" daemon [_thread_blocked, id=4752, stack(0x00007fd20f712000,0x00007fd20f813000)]
  0x00007fd4400b7800 JavaThread "elasticsearch[es-node-3-43][flush][T#62]" daemon [_thread_blocked, id=4747, stack(0x00007fd03636e000,0x00007fd03646f000)]
  0x00007fd2e0041800 JavaThread "elasticsearch[es-node-3-43][listener][T#3]" daemon [_thread_blocked, id=5834, stack(0x00007f7eb0e2c000,0x00007f7eb0f2d000)]
  0x00007fd340029000 JavaThread "elasticsearch[es-node-3-43][listener][T#2]" daemon [_thread_blocked, id=5543, stack(0x00007f7eafab8000,0x00007f7eafbb9000)]
  0x00007fd340037000 JavaThread "elasticsearch[es-node-3-43][listener][T#1]" daemon [_thread_blocked, id=5542, stack(0x00007fd324bf9000,0x00007fd324cfa000)]
  0x00007fd2cc023800 JavaThread "elasticsearch[es-node-3-43][generic][T#12]" daemon [_thread_blocked, id=27109, stack(0x00007f7eae2d0000,0x00007f7eae3d1000)]
  0x00007fd330027800 JavaThread "elasticsearch[es-node-3-43][generic][T#11]" daemon [_thread_blocked, id=30584, stack(0x00007fd0de522000,0x00007fd0de623000)]
  0x00007fd21c039800 JavaThread "elasticsearch[es-node-3-43][http_server_worker][T#32]" daemon [_thread_in_native, id=1505, stack(0x00007f7eb5b4d000,0x00007f7eb5c4e000)]
  0x00007fd21c038000 JavaThread "elasticsearch[es-node-3-43][http_server_worker][T#31]" daemon [_thread_in_native, id=1503, stack(0x00007f7eb5c4e000,0x00007f7eb5d4f000)]
  0x00007fd21c036000 JavaThread "elasticsearch[es-node-3-43][http_server_worker][T#30]" daemon [_thread_in_native, id=1502, stack(0x00007f7eb5d4f000,0x00007f7eb5e50000)]
  0x00007fd21c034000 JavaThread "elasticsearch[es-node-3-43][http_server_worker][T#29]" daemon [_thread_in_native, id=1501, stack(0x00007f7eb5e50000,0x00007f7eb5f51000)]
  0x00007fd21c032800 JavaThread "elasticsearch[es-node-3-43][http_server_worker][T#28]" daemon [_thread_in_native, id=1500, stack(0x00007f7eb5f51000,0x00007f7eb6052000)]
  0x00007fd21c030800 JavaThread "elasticsearch[es-node-3-43][http_server_worker][T#27]" daemon [_thread_in_native, id=1499, stack(0x00007f7eb6052000,0x00007f7eb6153000)]
  0x00007fd21c02f000 JavaThread "elasticsearch[es-node-3-43][http_server_worker][T#26]" daemon [_thread_in_native, id=1498, stack(0x00007f7eb6153000,0x00007f7eb6254000)]
  0x00007fd21c02d000 JavaThread "elasticsearch[es-node-3-43][http_server_worker][T#25]" daemon [_thread_in_native, id=1497, stack(0x00007f7eb6254000,0x00007f7eb6355000)]
  0x00007fd21c02b000 JavaThread "elasticsearch[es-node-3-43][http_server_worker][T#24]" daemon [_thread_in_native, id=1496, stack(0x00007f7eb6355000,0x00007f7eb6456000)]
  0x00007fd21c029800 JavaThread "elasticsearch[es-node-3-43][http_server_worker][T#23]" daemon [_thread_in_native, id=1495, stack(0x00007f7eb6456000,0x00007f7eb6557000)]
  0x00007fd21c027800 JavaThread "elasticsearch[es-node-3-43][http_server_worker][T#22]" daemon [_thread_in_native, id=1494, stack(0x00007f7eb6557000,0x00007f7eb6658000)]
  0x00007fd21c026000 JavaThread "elasticsearch[es-node-3-43][http_server_worker][T#21]" daemon [_thread_in_native, id=1493, stack(0x00007f7eb6658000,0x00007f7eb6759000)]
  0x00007fd21c024000 JavaThread "elasticsearch[es-node-3-43][http_server_worker][T#20]" daemon [_thread_in_native, id=1492, stack(0x00007f7eb6759000,0x00007f7eb685a000)]
  0x00007fd21c022000 JavaThread "elasticsearch[es-node-3-43][http_server_worker][T#19]" daemon [_thread_in_native, id=1490, stack(0x00007fd1f3a61000,0x00007fd1f3b62000)]
  0x00007fd21c021000 JavaThread "elasticsearch[es-node-3-43][http_server_worker][T#18]" daemon [_thread_in_native, id=1489, stack(0x00007fd20e2c1000,0x00007fd20e3c2000)]
  0x00007fd21c020000 JavaThread "elasticsearch[es-node-3-43][http_server_worker][T#17]" daemon [_thread_in_native, id=1488, stack(0x00007fd2fccfa000,0x00007fd2fcdfb000)]
  0x00007fd21c01c800 JavaThread "elasticsearch[es-node-3-43][http_server_worker][T#16]" daemon [_thread_in_native, id=1486, stack(0x00007fd2fc2f0000,0x00007fd2fc3f1000)]
  0x00007fd39001d800 JavaThread "elasticsearch[es-node-3-43][refresh][T#8]" daemon [_thread_blocked, id=28919, stack(0x00007fd2fc3f1000,0x00007fd2fc4f2000)]
  0x00007fd21c019800 JavaThread "elasticsearch[es-node-3-43][http_server_worker][T#15]" daemon [_thread_in_native, id=28380, stack(0x00007fd2fc5f3000,0x00007fd2fc6f4000)]
  0x00007fd390019000 JavaThread "elasticsearch[es-node-3-43][refresh][T#7]" daemon [_thread_blocked, id=28133, stack(0x00007fd3242f0000,0x00007fd3243f1000)]
  0x00007fd390018000 JavaThread "elasticsearch[es-node-3-43][refresh][T#6]" daemon [_thread_blocked, id=27907, stack(0x00007fd2fcefc000,0x00007fd2fcffd000)]
  0x00007fd21c01b800 JavaThread "elasticsearch[es-node-3-43][http_server_worker][T#14]" daemon [_thread_in_native, id=25572, stack(0x00007fd2fc4f2000,0x00007fd2fc5f3000)]
  0x00007fd21c01b000 JavaThread "elasticsearch[es-node-3-43][http_server_worker][T#13]" daemon [_thread_in_native, id=25346, stack(0x00007fd2fc9f7000,0x00007fd2fcaf8000)]
  0x00007fd21c018800 JavaThread "elasticsearch[es-node-3-43][http_server_worker][T#12]" daemon [_thread_in_native, id=24928, stack(0x00007fd210ea7000,0x00007fd210fa8000)]
  0x00007fd21c017800 JavaThread "elasticsearch[es-node-3-43][http_server_worker][T#11]" daemon [_thread_in_native, id=24927, stack(0x00007fcd97766000,0x00007fcd97867000)]
  0x00007fd21c016000 JavaThread "elasticsearch[es-node-3-43][http_server_worker][T#10]" daemon [_thread_in_native, id=24925, stack(0x00007fcd97a69000,0x00007fcd97b6a000)]
  0x00007fd21c015000 JavaThread "elasticsearch[es-node-3-43][http_server_worker][T#9]" daemon [_thread_in_native, id=24924, stack(0x00007fd3960f7000,0x00007fd3961f8000)]
  0x00007fd21c014000 JavaThread "elasticsearch[es-node-3-43][http_server_worker][T#8]" daemon [_thread_in_native, id=24922, stack(0x00007fd2110a9000,0x00007fd2111aa000)]
  0x00007fd21c013800 JavaThread "elasticsearch[es-node-3-43][http_server_worker][T#7]" daemon [_thread_in_native, id=24921, stack(0x00007fd1fb37d000,0x00007fd1fb47e000)]
  0x00007fd21c012800 JavaThread "elasticsearch[es-node-3-43][http_server_worker][T#6]" daemon [_thread_in_native, id=24920, stack(0x00007fcd39b9a000,0x00007fcd39c9b000)]
  0x00007fd21c012000 JavaThread "elasticsearch[es-node-3-43][http_server_worker][T#5]" daemon [_thread_in_native, id=24918, stack(0x00007fd210da6000,0x00007fd210ea7000)]
  0x00007fd21c00d000 JavaThread "elasticsearch[es-node-3-43][http_server_worker][T#4]" daemon [_thread_in_native, id=24917, stack(0x00007fcd39d9c000,0x00007fcd39e9d000)]
  0x00007fd30c017000 JavaThread "elasticsearch[es-node-3-43][write][T#16]" daemon [_thread_blocked, id=24916, stack(0x00007fd2112ab000,0x00007fd2113ac000)]
  0x00007fd2b401a800 JavaThread "elasticsearch[es-node-3-43][write][T#15]" daemon [_thread_in_Java, id=24915, stack(0x00007fcd21aee000,0x00007fcd21bef000)]
  0x00007fd30c016800 JavaThread "elasticsearch[es-node-3-43][write][T#14]" daemon [_thread_blocked, id=24914, stack(0x00007fd210ba4000,0x00007fd210ca5000)]
  0x00007fd344026000 JavaThread "elasticsearch[es-node-3-43][write][T#13]" daemon [_thread_blocked, id=24913, stack(0x00007fd210aa3000,0x00007fd210ba4000)]
  0x00007fd2dc01a000 JavaThread "elasticsearch[es-node-3-43][get][T#16]" daemon [_thread_blocked, id=24521, stack(0x00007fd210ca5000,0x00007fd210da6000)]
  0x00007fd34c015000 JavaThread "elasticsearch[es-node-3-43][write][T#12]" daemon [_thread_blocked, id=24518, stack(0x00007fd20e11c000,0x00007fd20e21d000)]
  0x00007fd2d401c800 JavaThread "elasticsearch[es-node-3-43][get][T#15]" daemon [_thread_blocked, id=24517, stack(0x00007fd2113ac000,0x00007fd2114ad000)]
  0x00007fd2e0030800 JavaThread "elasticsearch[es-node-3-43][get][T#14]" daemon [_thread_blocked, id=24516, stack(0x00007fcd39c9b000,0x00007fcd39d9c000)]
  0x00007fd2d401b800 JavaThread "elasticsearch[es-node-3-43][get][T#13]" daemon [_thread_blocked, id=24513, stack(0x00007fcd97c6b000,0x00007fcd97d6c000)]
  0x00007fd304017000 JavaThread "elasticsearch[es-node-3-43][get][T#12]" daemon [_thread_blocked, id=24511, stack(0x00007fcd219ed000,0x00007fcd21aee000)]
  0x00007fd304016000 JavaThread "elasticsearch[es-node-3-43][get][T#11]" daemon [_thread_blocked, id=24510, stack(0x00007fd20e01b000,0x00007fd20e11c000)]
  0x00007fd304014800 JavaThread "elasticsearch[es-node-3-43][get][T#10]" daemon [_thread_blocked, id=24509, stack(0x00007fd210fa8000,0x00007fd2110a9000)]
  0x00007fd304013800 JavaThread "elasticsearch[es-node-3-43][get][T#9]" daemon [_thread_blocked, id=24508, stack(0x00007fd21069f000,0x00007fd2107a0000)]
  0x00007fd2dc019000 JavaThread "elasticsearch[es-node-3-43][get][T#8]" daemon [_thread_blocked, id=24507, stack(0x00007fd3247f5000,0x00007fd3248f6000)]
  0x00007fd2dc018000 JavaThread "elasticsearch[es-node-3-43][get][T#7]" daemon [_thread_blocked, id=24506, stack(0x00007fd3246f4000,0x00007fd3247f5000)]
  0x00007fd2f0014800 JavaThread "elasticsearch[es-node-3-43][get][T#5]" daemon [_thread_blocked, id=24505, stack(0x00007fd3249f7000,0x00007fd324af8000)]
  0x00007fd2f8013000 JavaThread "elasticsearch[es-node-3-43][get][T#6]" daemon [_thread_blocked, id=24504, stack(0x00007fd3248f6000,0x00007fd3249f7000)]
  0x00007fd2d401a000 JavaThread "elasticsearch[es-node-3-43][get][T#4]" daemon [_thread_blocked, id=24503, stack(0x00007fd2108a1000,0x00007fd2109a2000)]
  0x00007fd2d4019000 JavaThread "elasticsearch[es-node-3-43][get][T#3]" daemon [_thread_blocked, id=24502, stack(0x00007fcd97e6d000,0x00007fcd97f6e000)]
  0x00007fd304012800 JavaThread "elasticsearch[es-node-3-43][get][T#2]" daemon [_thread_blocked, id=24501, stack(0x00007fcd97867000,0x00007fcd97968000)]
  0x00007fd2dc017000 JavaThread "elasticsearch[es-node-3-43][get][T#1]" daemon [_thread_blocked, id=24500, stack(0x00007fcd97b6a000,0x00007fcd97c6b000)]
  0x00007fd2cc01c800 JavaThread "elasticsearch[es-node-3-43][write][T#11]" daemon [_thread_blocked, id=24372, stack(0x00007fcd97968000,0x00007fcd97a69000)]
  0x00007fd2cc01b800 JavaThread "elasticsearch[es-node-3-43][write][T#10]" daemon [_thread_blocked, id=24111, stack(0x00007fd2107a0000,0x00007fd2108a1000)]
  0x00007fd2cc013800 JavaThread "elasticsearch[es-node-3-43][write][T#9]" daemon [_thread_blocked, id=23899, stack(0x00007fcd97d6c000,0x00007fcd97e6d000)]
  0x00007fd2dc015800 JavaThread "elasticsearch[es-node-3-43][search][T#25]" daemon [_thread_blocked, id=23898, stack(0x00007fcef4c59000,0x00007fcef4d5a000)]
  0x00007fd314014800 JavaThread "elasticsearch[es-node-3-43][search][T#24]" daemon [_thread_blocked, id=23897, stack(0x00007fd2111aa000,0x00007fd2112ab000)]
  0x00007fd2cc01b000 JavaThread "elasticsearch[es-node-3-43][write][T#8]" daemon [_thread_blocked, id=23718, stack(0x00007fd2fc6f4000,0x00007fd2fc7f5000)]
  0x00007fd2e8016000 JavaThread "elasticsearch[es-node-3-43][search][T#23]" daemon [_thread_blocked, id=23717, stack(0x00007fd2fc0ee000,0x00007fd2fc1ef000)]
  0x00007fd2ec017800 JavaThread "elasticsearch[es-node-3-43][search][T#22]" daemon [_thread_blocked, id=23716, stack(0x00007fd3240ee000,0x00007fd3241ef000)]
  0x00007fd2cc019000 JavaThread "elasticsearch[es-node-3-43][write][T#7]" daemon [_thread_blocked, id=23580, stack(0x00007fd324dfb000,0x00007fd324efc000)]
  0x00007fd2d8016800 JavaThread "elasticsearch[es-node-3-43][search][T#21]" daemon [_thread_blocked, id=23579, stack(0x00007fd3241ef000,0x00007fd3242f0000)]
  0x00007fd2d8015800 JavaThread "elasticsearch[es-node-3-43][search][T#20]" daemon [_thread_blocked, id=23578, stack(0x00007fd32c0f0000,0x00007fd32c1f1000)]
  0x00007fd2e4020000 JavaThread "elasticsearch[es-node-3-43][search][T#19]" daemon [_thread_blocked, id=23577, stack(0x00007f83ee92b000,0x00007f83eea2c000)]
  0x00007fd2e401f000 JavaThread "elasticsearch[es-node-3-43][search][T#18]" daemon [_thread_blocked, id=23576, stack(0x00007f83ee82a000,0x00007f83ee92b000)]
  0x00007fd2cc018000 JavaThread "elasticsearch[es-node-3-43][write][T#6]" daemon [_thread_blocked, id=23342, stack(0x00007fd2fcaf8000,0x00007fd2fcbf9000)]
=>0x00007fd2dc014800 JavaThread "elasticsearch[es-node-3-43][search][T#17]" daemon [_thread_in_Java, id=23341, stack(0x00007fd3244f2000,0x00007fd3245f3000)]
  0x00007fd2d8014800 JavaThread "elasticsearch[es-node-3-43][search][T#16]" daemon [_thread_blocked, id=23340, stack(0x00007fd2fcdfb000,0x00007fd2fcefc000)]
  0x00007fd2e401e000 JavaThread "elasticsearch[es-node-3-43][search][T#15]" daemon [_thread_blocked, id=23178, stack(0x00007fd2fc1ef000,0x00007fd2fc2f0000)]
  0x00007fd34c003800 JavaThread "elasticsearch[es-node-3-43][write][T#5]" daemon [_thread_blocked, id=23177, stack(0x00007fd324efc000,0x00007fd324ffd000)]
  0x00007fd2e8015000 JavaThread "elasticsearch[es-node-3-43][search][T#14]" daemon [_thread_blocked, id=23176, stack(0x00007fd324af8000,0x00007fd324bf9000)]
  0x00007fd2d4018000 JavaThread "elasticsearch[es-node-3-43][search][T#13]" daemon [_thread_blocked, id=23175, stack(0x00007fd3245f3000,0x00007fd3246f4000)]
  0x00007fd2d8012000 JavaThread "elasticsearch[es-node-3-43][search][T#12]" daemon [_thread_blocked, id=23174, stack(0x00007fd324cfa000,0x00007fd324dfb000)]
  0x00007fd32002a800 JavaThread "elasticsearch[es-node-3-43][ccr][T#32]" daemon [_thread_blocked, id=23117, stack(0x00007fd2fc8f6000,0x00007fd2fc9f7000)]
  0x00007fd33000b000 JavaThread "elasticsearch[es-node-3-43][ccr][T#31]" daemon [_thread_blocked, id=23085, stack(0x00007fd2fcbf9000,0x00007fd2fccfa000)]
  0x00007fd33c04f000 JavaThread "elasticsearch[es-node-3-43][ccr][T#30]" daemon [_thread_blocked, id=23080, stack(0x00007fd2fc7f5000,0x00007fd2fc8f6000)]
  0x00007fd33006a800 JavaThread "elasticsearch[es-node-3-43][ccr][T#29]" daemon [_thread_blocked, id=23002, stack(0x00007f6c89c9e000,0x00007f6c89d9f000)]
  0x00007fd33803b000 JavaThread "elasticsearch[es-node-3-43][ccr][T#28]" daemon [_thread_blocked, id=22937, stack(0x00007f6c89d9f000,0x00007f6c89ea0000)]
  0x00007fd2d4016000 JavaThread "elasticsearch[es-node-3-43][search][T#11]" daemon [_thread_blocked, id=22910, stack(0x00007f6c89ea0000,0x00007f6c89fa1000)]
  0x00007fd2cc016000 JavaThread "elasticsearch[es-node-3-43][write][T#4]" daemon [_thread_in_Java, id=22909, stack(0x00007f6c89fa1000,0x00007f6c8a0a2000)]
  0x00007fd2e8013800 JavaThread "elasticsearch[es-node-3-43][search][T#10]" daemon [_thread_blocked, id=22908, stack(0x00007f6c8a0a2000,0x00007f6c8a1a3000)]
  0x00007fd330068800 JavaThread "elasticsearch[es-node-3-43][ccr][T#27]" daemon [_thread_blocked, id=22907, stack(0x00007f6c8a1a3000,0x00007f6c8a2a4000)]
  0x00007fd2e401c000 JavaThread "elasticsearch[es-node-3-43][search][T#9]" daemon [_thread_blocked, id=22902, stack(0x00007f6c8a2a6000,0x00007f6c8a3a7000)]
  0x00007fd2e002c800 JavaThread "elasticsearch[es-node-3-43][search][T#8]" daemon [_thread_blocked, id=22901, stack(0x00007f6c8a3a7000,0x00007f6c8a4a8000)]
  0x00007fd320028800 JavaThread "elasticsearch[es-node-3-43][write][T#3]" daemon [_thread_blocked, id=22900, stack(0x00007f6c8a4a8000,0x00007f6c8a5a9000)]
  0x00007fd34404f000 JavaThread "elasticsearch[es-node-3-43][ccr][T#26]" daemon [_thread_blocked, id=22899, stack(0x00007f6c8a5a9000,0x00007f6c8a6aa000)]
  0x00007fd314011800 JavaThread "elasticsearch[es-node-3-43][search][T#7]" daemon [_thread_blocked, id=22898, stack(0x00007f6c8a6aa000,0x00007f6c8a7ab000)]
  0x00007fd318028000 JavaThread "elasticsearch[es-node-3-43][ccr][T#25]" daemon [_thread_blocked, id=22895, stack(0x00007f6c8a7ab000,0x00007f6c8a8ac000)]
  0x00007fd34404d000 JavaThread "elasticsearch[es-node-3-43][ccr][T#24]" daemon [_thread_blocked, id=22894, stack(0x00007f6c8a8d8000,0x00007f6c8a9d9000)]
  0x00007fd330067800 JavaThread "elasticsearch[es-node-3-43][ccr][T#23]" daemon [_thread_blocked, id=22893, stack(0x00007f6c8aa61000,0x00007f6c8ab62000)]
  0x00007fd318026800 JavaThread "elasticsearch[es-node-3-43][ccr][T#22]" daemon [_thread_blocked, id=22765, stack(0x00007f6c8ab62000,0x00007f6c8ac63000)]
  0x00007fd3300ad800 JavaThread "elasticsearch[es-node-3-43][ccr][T#21]" daemon [_thread_blocked, id=22756, stack(0x00007f6c8b1ea000,0x00007f6c8b2eb000)]
  0x00007fd34404b800 JavaThread "elasticsearch[es-node-3-43][ccr][T#20]" daemon [_thread_blocked, id=22687, stack(0x00007f6e9ceb9000,0x00007f6e9cfba000)]
  0x00007fd33c04c800 JavaThread "elasticsearch[es-node-3-43][ccr][T#19]" daemon [_thread_blocked, id=22666, stack(0x00007f6e9cfba000,0x00007f6e9d0bb000)]
  0x00007fd318024800 JavaThread "elasticsearch[es-node-3-43][ccr][T#18]" daemon [_thread_blocked, id=22665, stack(0x00007f6eab77a000,0x00007f6eab87b000)]
  0x00007fd338038000 JavaThread "elasticsearch[es-node-3-43][ccr][T#17]" daemon [_thread_blocked, id=22664, stack(0x00007f6eb8037000,0x00007f6eb8138000)]
  0x00007fd390015000 JavaThread "elasticsearch[es-node-3-43][refresh][T#5]" daemon [_thread_blocked, id=22528, stack(0x00007f74f7ac7000,0x00007f74f7bc8000)]
  0x00007fd318023000 JavaThread "elasticsearch[es-node-3-43][ccr][T#16]" daemon [_thread_blocked, id=22502, stack(0x00007f7aef6db000,0x00007f7aef7dc000)]
  0x00007fd2cc014800 JavaThread "elasticsearch[es-node-3-43][write][T#2]" daemon [_thread_in_vm_trans, id=22493, stack(0x00007f7b21a11000,0x00007f7b21b12000)]
  0x00007fd2d4015000 JavaThread "elasticsearch[es-node-3-43][search][T#6]" daemon [_thread_blocked, id=22492, stack(0x00007f7b21d12000,0x00007f7b21e13000)]
  0x00007fd2f0012800 JavaThread "elasticsearch[es-node-3-43][write][T#1]" daemon [_thread_blocked, id=22491, stack(0x00007f7b21e13000,0x00007f7b21f14000)]
  0x00007fd2e401a800 JavaThread "elasticsearch[es-node-3-43][search][T#5]" daemon [_thread_blocked, id=22490, stack(0x00007f7b21f14000,0x00007f7b22015000)]
  0x00007fd2ec019000 JavaThread "elasticsearch[es-node-3-43][search][T#4]" daemon [_thread_blocked, id=22489, stack(0x00007f7b22015000,0x00007f7b22116000)]
  0x00007fd2dc013000 JavaThread "elasticsearch[es-node-3-43][search][T#3]" daemon [_thread_blocked, id=22488, stack(0x00007f7b22116000,0x00007f7b22217000)]
  0x00007fd2e4017800 JavaThread "elasticsearch[es-node-3-43][search][T#2]" daemon [_thread_blocked, id=22487, stack(0x00007f7eb0b56000,0x00007f7eb0c57000)]
  0x00007fd2e0013000 JavaThread "elasticsearch[es-node-3-43][search][T#1]" daemon [_thread_blocked, id=22486, stack(0x00007f7eb8bbd000,0x00007f7eb8cbe000)]
  0x00007fd344049800 JavaThread "elasticsearch[es-node-3-43][ccr][T#15]" daemon [_thread_blocked, id=22483, stack(0x00007f7eceae2000,0x00007f7ecebe3000)]
  0x00007fd33004a000 JavaThread "elasticsearch[es-node-3-43][ccr][T#14]" daemon [_thread_blocked, id=22478, stack(0x00007f7fd8ae9000,0x00007f7fd8bea000)]
  0x00007fd320027000 JavaThread "elasticsearch[es-node-3-43][ccr][T#13]" daemon [_thread_blocked, id=22477, stack(0x00007f83ee628000,0x00007f83ee729000)]
  0x00007fd344047800 JavaThread "elasticsearch[es-node-3-43][ccr][T#12]" daemon [_thread_blocked, id=22476, stack(0x00007f83ee729000,0x00007f83ee82a000)]
  0x00007fd350013800 JavaThread "elasticsearch[es-node-3-43][warmer][T#1]" daemon [_thread_blocked, id=22473, stack(0x00007f83eea2c000,0x00007f83eeb2d000)]
  0x00007fd338034800 JavaThread "elasticsearch[es-node-3-43][ccr][T#11]" daemon [_thread_blocked, id=22472, stack(0x00007f8b039de000,0x00007f8b03adf000)]
  0x00007fd320026000 JavaThread "elasticsearch[es-node-3-43][ccr][T#10]" daemon [_thread_blocked, id=22343, stack(0x00007fa10c9d3000,0x00007fa10cad4000)]
  0x00007fd344046000 JavaThread "elasticsearch[es-node-3-43][ccr][T#9]" daemon [_thread_blocked, id=22341, stack(0x00007fa5cb7e4000,0x00007fa5cb8e5000)]
  0x00007fd338033800 JavaThread "elasticsearch[es-node-3-43][ccr][T#8]" daemon [_thread_blocked, id=22297, stack(0x00007fd395ef5000,0x00007fd395ff6000)]
  0x00007fd33c037000 JavaThread "elasticsearch[es-node-3-43][ccr][T#7]" daemon [_thread_blocked, id=22252, stack(0x00007fd395cf3000,0x00007fd395df4000)]
  0x00007fd318021800 JavaThread "elasticsearch[es-node-3-43][ccr][T#6]" daemon [_thread_blocked, id=22251, stack(0x00007fd395af1000,0x00007fd395bf2000)]
  0x00007fd38400b000 JavaThread "elasticsearch[es-node-3-43][generic][T#10]" daemon [_thread_blocked, id=22249, stack(0x00007fd395bf2000,0x00007fd395cf3000)]
  0x00007fd390012000 JavaThread "elasticsearch[es-node-3-43][generic][T#9]" daemon [_thread_blocked, id=22248, stack(0x00007fd395ff6000,0x00007fd3960f7000)]
  0x00007fd340003000 JavaThread "elasticsearch[es-node-3-43][generic][T#8]" daemon [_thread_blocked, id=22247, stack(0x00007fd3959f0000,0x00007fd395af1000)]
  0x00007fd390010800 JavaThread "elasticsearch[es-node-3-43][generic][T#7]" daemon [_thread_blocked, id=22246, stack(0x00007fd3961f8000,0x00007fd3962f9000)]
  0x00007fd338032000 JavaThread "elasticsearch[es-node-3-43][ccr][T#5]" daemon [_thread_blocked, id=22245, stack(0x00007fbe76aa5000,0x00007fbe76ba6000)]
  0x00007fd3e0001000 JavaThread "Attach Listener" daemon [_thread_blocked, id=22239, stack(0x00007fbe76ba6000,0x00007fbe76ca7000)]
  0x00007fd344043000 JavaThread "elasticsearch[es-node-3-43][ccr][T#4]" daemon [_thread_blocked, id=22054, stack(0x00007fca70e1f000,0x00007fca70f20000)]
  0x00007fd33800c800 JavaThread "elasticsearch[es-node-3-43][ccr][T#3]" daemon [_thread_blocked, id=22005, stack(0x00007fcae10e0000,0x00007fcae11e1000)]
  0x00007fd39000f000 JavaThread "elasticsearch[es-node-3-43][refresh][T#4]" daemon [_thread_blocked, id=22004, stack(0x00007fcaf162b000,0x00007fcaf172c000)]
  0x00007fd39000d000 JavaThread "elasticsearch[es-node-3-43][refresh][T#3]" daemon [_thread_blocked, id=21988, stack(0x00007fcff2d64000,0x00007fcff2e65000)]
  0x00007fd2c4026000 JavaThread "elasticsearch[es-node-3-43][generic][T#6]" daemon [_thread_blocked, id=21986, stack(0x00007fd01e436000,0x00007fd01e537000)]
  0x00007fd338018000 JavaThread "elasticsearch[es-node-3-43][ccr][T#2]" daemon [_thread_blocked, id=21983, stack(0x00007fd01eedb000,0x00007fd01efdc000)]
  0x00007fd39000b800 JavaThread "elasticsearch[es-node-3-43][refresh][T#2]" daemon [_thread_blocked, id=21982, stack(0x00007fd1f9056000,0x00007fd1f9157000)]
  0x00007fd33c03d000 JavaThread "elasticsearch[es-node-3-43][fetch_shard_store][T#11]" daemon [_thread_blocked, id=21975, stack(0x00007fd2109a2000,0x00007fd210aa3000)]
  0x00007fd39000a800 JavaThread "elasticsearch[es-node-3-43][refresh][T#1]" daemon [_thread_blocked, id=21964, stack(0x00007fd212b64000,0x00007fd212c65000)]
  0x00007fd34403a800 JavaThread "elasticsearch[es-node-3-43][ccr][T#1]" daemon [_thread_blocked, id=21963, stack(0x00007fd212c65000,0x00007fd212d66000)]
  0x00007fd21c008800 JavaThread "elasticsearch[es-node-3-43][http_server_worker][T#3]" daemon [_thread_in_native, id=21962, stack(0x00007fd22006f000,0x00007fd220170000)]
  0x00007fd21c002800 JavaThread "elasticsearch[es-node-3-43][http_server_worker][T#2]" daemon [_thread_in_native, id=21961, stack(0x00007fd27807f000,0x00007fd278180000)]
  0x00007fd47400b000 JavaThread "DestroyJavaVM" [_thread_blocked, id=21709, stack(0x00007fd47a94c000,0x00007fd47aa4d000)]
  0x00007fd4751e0800 JavaThread "elasticsearch[keepAlive/6.6.0]" [_thread_blocked, id=21960, stack(0x00007fd2d04f4000,0x00007fd2d05f5000)]
  0x00007fd474a29800 JavaThread "elasticsearch[es-node-3-43][http_server_worker][T#1]" daemon [_thread_in_native, id=21959, stack(0x00007fd2d05f5000,0x00007fd2d06f6000)]
  0x00007fd36030b000 JavaThread "elasticsearch[es-node-3-43][trigger_engine_scheduler][T#1]" daemon [_thread_blocked, id=21958, stack(0x00007fd2d06f6000,0x00007fd2d07f7000)]
  0x00007fd360009000 JavaThread "elasticsearch[es-node-3-43][management][T#5]" daemon [_thread_blocked, id=21957, stack(0x00007fd2d09f7000,0x00007fd2d0af8000)]
  0x00007fd360007000 JavaThread "elasticsearch[es-node-3-43][management][T#4]" daemon [_thread_blocked, id=21956, stack(0x00007fd2d0af8000,0x00007fd2d0bf9000)]
  0x00007fd360005800 JavaThread "elasticsearch[es-node-3-43][management][T#3]" daemon [_thread_blocked, id=21955, stack(0x00007fd2d0bf9000,0x00007fd2d0cfa000)]
  0x00007fd360003800 JavaThread "elasticsearch[es-node-3-43][management][T#2]" daemon [_thread_blocked, id=21953, stack(0x00007fd2d0cfa000,0x00007fd2d0dfb000)]
  0x00007fd360002000 JavaThread "elasticsearch[es-node-3-43][management][T#1]" daemon [_thread_blocked, id=21952, stack(0x00007fd2d0dfb000,0x00007fd2d0efc000)]
  0x00007fd33003a000 JavaThread "elasticsearch[es-node-3-43][fetch_shard_started][T#32]" daemon [_thread_blocked, id=21950, stack(0x00007fd2d0efc000,0x00007fd2d0ffd000)]
  0x00007fd37c02d000 JavaThread "elasticsearch[es-node-3-43][transport_worker][T#32]" daemon [_thread_in_native, id=21917, stack(0x00007fd32c3f1000,0x00007fd32c4f2000)]
  0x00007fd37c02b800 JavaThread "elasticsearch[es-node-3-43][transport_worker][T#31]" daemon [_thread_in_native, id=21916, stack(0x00007fd32c4f2000,0x00007fd32c5f3000)]
  0x00007fd37c029800 JavaThread "elasticsearch[es-node-3-43][transport_worker][T#30]" daemon [_thread_in_native, id=21915, stack(0x00007fd32c5f3000,0x00007fd32c6f4000)]
  0x00007fd37c028000 JavaThread "elasticsearch[es-node-3-43][transport_worker][T#29]" daemon [_thread_in_native, id=21914, stack(0x00007fd32c6f4000,0x00007fd32c7f5000)]
  0x00007fd390007000 JavaThread "elasticsearch[es-node-3-43][generic][T#5]" daemon [_thread_blocked, id=21913, stack(0x00007fd32c7f5000,0x00007fd32c8f6000)]
  0x00007fd37c026800 JavaThread "elasticsearch[es-node-3-43][transport_worker][T#28]" daemon [_thread_blocked, id=21912, stack(0x00007fd32c8f6000,0x00007fd32c9f7000)]
  0x00007fd37c024000 JavaThread "elasticsearch[es-node-3-43][transport_worker][T#27]" daemon [_thread_in_native, id=21911, stack(0x00007fd32c9f7000,0x00007fd32caf8000)]
  0x00007fd378006000 JavaThread "elasticsearch[es-node-3-43][transport_worker][T#26]" daemon [_thread_in_native, id=21910, stack(0x00007fd32caf8000,0x00007fd32cbf9000)]
  0x00007fd37c022000 JavaThread "elasticsearch[es-node-3-43][transport_worker][T#25]" daemon [_thread_in_native, id=21909, stack(0x00007fd32cbf9000,0x00007fd32ccfa000)]
  0x00007fd37c020800 JavaThread "elasticsearch[es-node-3-43][transport_worker][T#24]" daemon [_thread_in_native, id=21908, stack(0x00007fd32ccfa000,0x00007fd32cdfb000)]
  0x00007fd37c01e800 JavaThread "elasticsearch[es-node-3-43][transport_worker][T#23]" daemon [_thread_in_native, id=21907, stack(0x00007fd32cdfb000,0x00007fd32cefc000)]
  0x00007fd37c01d000 JavaThread "elasticsearch[es-node-3-43][transport_worker][T#22]" daemon [_thread_in_native, id=21906, stack(0x00007fd32cefc000,0x00007fd32cffd000)]
  0x00007fd37c01b000 JavaThread "elasticsearch[es-node-3-43][transport_worker][T#21]" daemon [_thread_in_native, id=21905, stack(0x00007fd3940d9000,0x00007fd3941da000)]
  0x00007fd37c019800 JavaThread "elasticsearch[es-node-3-43][transport_worker][T#20]" daemon [_thread_in_native, id=21904, stack(0x00007fd3941da000,0x00007fd3942db000)]
  0x00007fd37c017800 JavaThread "elasticsearch[es-node-3-43][masterService#updateTask][T#1]" daemon [_thread_blocked, id=21903, stack(0x00007fd3942db000,0x00007fd3943dc000)]
  0x00007fd378005000 JavaThread "elasticsearch[es-node-3-43][transport_worker][T#19]" daemon [_thread_in_native, id=21902, stack(0x00007fd3943dc000,0x00007fd3944dd000)]
  0x00007fd358008000 JavaThread "elasticsearch[es-node-3-43][transport_worker][T#17]" daemon [_thread_in_native, id=21901, stack(0x00007fd3944dd000,0x00007fd3945de000)]
  0x00007fd36800e800 JavaThread "elasticsearch[es-node-3-43][transport_worker][T#16]" daemon [_thread_in_native, id=21900, stack(0x00007fd3945de000,0x00007fd3946df000)]
  0x00007fd350010000 JavaThread "elasticsearch[es-node-3-43][transport_worker][T#18]" daemon [_thread_blocked, id=21899, stack(0x00007fd3946df000,0x00007fd3947e0000)]
  0x00007fd37000d000 JavaThread "elasticsearch[es-node-3-43][transport_worker][T#15]" daemon [_thread_in_native, id=21898, stack(0x00007fd3947e0000,0x00007fd3948e1000)]
  0x00007fd35c006000 JavaThread "elasticsearch[es-node-3-43][transport_worker][T#14]" daemon [_thread_in_native, id=21897, stack(0x00007fd3948e1000,0x00007fd3949e2000)]
  0x00007fd36800c800 JavaThread "elasticsearch[es-node-3-43][transport_worker][T#11]" daemon [_thread_in_native, id=21896, stack(0x00007fd3949e2000,0x00007fd394ae3000)]
  0x00007fd37000b800 JavaThread "elasticsearch[es-node-3-43][transport_worker][T#13]" daemon [_thread_in_native_trans, id=21895, stack(0x00007fd394ae3000,0x00007fd394be4000)]
  0x00007fd35000e000 JavaThread "elasticsearch[es-node-3-43][transport_worker][T#12]" daemon [_thread_in_native, id=21894, stack(0x00007fd394be4000,0x00007fd394ce5000)]
  0x00007fd378003800 JavaThread "elasticsearch[es-node-3-43][transport_worker][T#10]" daemon [_thread_in_native, id=21893, stack(0x00007fd394ce5000,0x00007fd394de6000)]
  0x00007fd390004000 JavaThread "elasticsearch[es-node-3-43][generic][T#4]" daemon [_thread_blocked, id=21866, stack(0x00007fd394de6000,0x00007fd394ee7000)]
  0x00007fd36c002800 JavaThread "elasticsearch[es-node-3-43][transport_worker][T#8]" daemon [_thread_in_native, id=21860, stack(0x00007fd3950e7000,0x00007fd3951e8000)]
  0x00007fd374002800 JavaThread "elasticsearch[es-node-3-43][transport_worker][T#3]" daemon [_thread_in_native, id=21859, stack(0x00007fd3951e8000,0x00007fd3952e9000)]
  0x00007fd35c001800 JavaThread "elasticsearch[es-node-3-43][transport_worker][T#4]" daemon [_thread_in_native, id=21858, stack(0x00007fd3952e9000,0x00007fd3953ea000)]
  0x00007fd35000b000 JavaThread "elasticsearch[es-node-3-43][transport_worker][T#2]" daemon [_thread_in_native, id=21857, stack(0x00007fd3953ea000,0x00007fd3954eb000)]
  0x00007fd370007000 JavaThread "elasticsearch[es-node-3-43][transport_worker][T#7]" daemon [_thread_in_native, id=21856, stack(0x00007fd3954eb000,0x00007fd3955ec000)]
  0x00007fd364002800 JavaThread "elasticsearch[es-node-3-43][transport_worker][T#9]" daemon [_thread_in_native, id=21855, stack(0x00007fd3955ec000,0x00007fd3956ed000)]
  0x00007fd368007000 JavaThread "elasticsearch[es-node-3-43][transport_worker][T#6]" daemon [_thread_blocked, id=21854, stack(0x00007fd3956ed000,0x00007fd3957ee000)]
  0x00007fd358002000 JavaThread "elasticsearch[es-node-3-43][transport_worker][T#5]" daemon [_thread_in_native, id=21853, stack(0x00007fd3957ee000,0x00007fd3958ef000)]
  0x00007fd37c013800 JavaThread "elasticsearch[es-node-3-43][generic][T#3]" daemon [_thread_blocked, id=21852, stack(0x00007fd3958ef000,0x00007fd3959f0000)]
  0x00007fd47691d800 JavaThread "elasticsearch[es-node-3-43][clusterApplierService#updateTask][T#1]" daemon [_thread_blocked, id=21847, stack(0x00007fd395df4000,0x00007fd395ef5000)]
  0x00007fd47690f800 JavaThread "elasticsearch[es-node-3-43][generic][T#2]" daemon [_thread_blocked, id=21842, stack(0x00007fd3962f9000,0x00007fd3963fa000)]
  0x00007fd4768f8800 JavaThread "elasticsearch[es-node-3-43][transport_worker][T#1]" daemon [_thread_in_native, id=21841, stack(0x00007fd3963fa000,0x00007fd3964fb000)]
  0x00007fd390001000 JavaThread "elasticsearch[es-node-3-43][generic][T#1]" daemon [_thread_blocked, id=21835, stack(0x00007fd396cfb000,0x00007fd396dfc000)]
  0x00007fd475e85000 JavaThread "ml-cpp-log-tail-thread" daemon [_thread_in_native, id=21829, stack(0x00007fd3973fc000,0x00007fd3974fd000)]
  0x00007fd475e49000 JavaThread "ticker-schedule-trigger-engine" daemon [_thread_blocked, id=21827, stack(0x00007fd3974fd000,0x00007fd3975fe000)]
  0x00007fd475dda800 JavaThread "Connection evictor" daemon [_thread_blocked, id=21826, stack(0x00007fd3977fe000,0x00007fd3978ff000)]
  0x00007fd475ce1000 JavaThread "elasticsearch[es-node-3-43][scheduler][T#1]" daemon [_thread_blocked, id=21815, stack(0x00007fd397aff000,0x00007fd397c00000)]
  0x00007fd475232000 JavaThread "elasticsearch[es-node-3-43][[timer]]" daemon [_thread_blocked, id=21809, stack(0x00007fd3a4f5a000,0x00007fd3a505b000)]
  0x00007fd474799000 JavaThread "process reaper" daemon [_thread_in_native, id=21782, stack(0x00007fd410066000,0x00007fd41009f000)]
  0x00007fd474196000 JavaThread "Service Thread" daemon [_thread_blocked, id=21779, stack(0x00007fd4106d4000,0x00007fd4107d5000)]
  0x00007fd474193000 JavaThread "C1 CompilerThread11" daemon [_thread_blocked, id=21778, stack(0x00007fd4107d5000,0x00007fd4108d6000)]
  0x00007fd474191000 JavaThread "C1 CompilerThread10" daemon [_thread_in_vm_trans, id=21777, stack(0x00007fd4108d6000,0x00007fd4109d7000)]
  0x00007fd47418f800 JavaThread "C1 CompilerThread9" daemon [_thread_blocked, id=21776, stack(0x00007fd4109d7000,0x00007fd410ad8000)]
  0x00007fd47418c800 JavaThread "C1 CompilerThread8" daemon [_thread_blocked, id=21775, stack(0x00007fd410ad8000,0x00007fd410bd9000)]
  0x00007fd47418a800 JavaThread "C2 CompilerThread7" daemon [_thread_blocked, id=21774, stack(0x00007fd410bd9000,0x00007fd410cda000)]
  0x00007fd474188000 JavaThread "C2 CompilerThread6" daemon [_thread_blocked, id=21773, stack(0x00007fd410cda000,0x00007fd410ddb000)]
  0x00007fd474186000 JavaThread "C2 CompilerThread5" daemon [_thread_blocked, id=21772, stack(0x00007fd410ddb000,0x00007fd410edc000)]
  0x00007fd474183800 JavaThread "C2 CompilerThread4" daemon [_thread_blocked, id=21771, stack(0x00007fd410edc000,0x00007fd410fdd000)]
  0x00007fd474179800 JavaThread "C2 CompilerThread3" daemon [_thread_blocked, id=21770, stack(0x00007fd410fdd000,0x00007fd4110de000)]
  0x00007fd474177000 JavaThread "C2 CompilerThread2" daemon [_thread_blocked, id=21769, stack(0x00007fd4110de000,0x00007fd4111df000)]
  0x00007fd474175000 JavaThread "C2 CompilerThread1" daemon [_thread_blocked, id=21768, stack(0x00007fd4111df000,0x00007fd4112e0000)]
  0x00007fd474173000 JavaThread "C2 CompilerThread0" daemon [_thread_blocked, id=21767, stack(0x00007fd4112e0000,0x00007fd4113e1000)]
  0x00007fd474170800 JavaThread "Signal Dispatcher" daemon [_thread_blocked, id=21766, stack(0x00007fd4113e1000,0x00007fd4114e2000)]
  0x00007fd47416f000 JavaThread "Surrogate Locker Thread (Concurrent GC)" daemon [_thread_blocked, id=21765, stack(0x00007fd4114e2000,0x00007fd4115e3000)]
  0x00007fd47413c800 JavaThread "Finalizer" daemon [_thread_blocked, id=21764, stack(0x00007fd4115e3000,0x00007fd4116e4000)]
  0x00007fd474137800 JavaThread "Reference Handler" daemon [_thread_blocked, id=21763, stack(0x00007fd4116e4000,0x00007fd4117e5000)]

Other Threads:
  0x00007fd474130000 VMThread [stack: 0x00007fd4117e5000,0x00007fd4118e6000] [id=21762]
  0x00007fd474199000 WatcherThread [stack: 0x00007fd4105d3000,0x00007fd4106d4000] [id=21780]

VM state:synchronizing (normal execution)

VM Mutex/Monitor currently owned by a thread:  ([mutex/lock_event])
[0x00007fd474008860] Safepoint_lock - owner thread: 0x00007fd474130000
[0x00007fd4740088e0] Threads_lock - owner thread: 0x00007fd474130000

Heap:
 par new generation   total 996800K, used 116295K [0x0000000280000000, 0x00000002c3990000, 0x00000002c3990000)
  eden space 886080K,   4% used [0x0000000280000000, 0x0000000282333f38, 0x00000002b6150000)
  from space 110720K,  72% used [0x00000002bcd70000, 0x00000002c1bcdde0, 0x00000002c3990000)
  to   space 110720K,   0% used [0x00000002b6150000, 0x00000002b6150000, 0x00000002bcd70000)
 concurrent mark-sweep generation total 20912576K, used 12725547K [0x00000002c3990000, 0x00000007c0000000, 0x00000007c0000000)
 Metaspace       used 91773K, capacity 99868K, committed 100984K, reserved 1136640K
  class space    used 11496K, capacity 13754K, committed 14048K, reserved 1048576K

Card table byte_map: [0x00007fd4354fe000,0x00007fd437eff000] byte_map_base: 0x00007fd4340fe000

Marking Bits: (CMSBitMap*) 0x00007fd4740d39c8
 Bits: [0x00007fd4140e6000, 0x00007fd427fffc00)

Mod Union Table: (CMSBitMap*) 0x00007fd4740d3a88
 Bits: [0x00007fd4342f4000, 0x00007fd4347f0670)

Polling page: 0x00007fd47aa55000

CodeCache: size=245760Kb used=86003Kb max_used=86091Kb free=159757Kb
 bounds [0x00007fd465000000, 0x00007fd46a4f0000, 0x00007fd474000000]
 total_blobs=23144 nmethods=22231 adapters=822
 compilation: enabled

Compilation events (10 events):
Event: 223703.474 Thread 0x00007fd474173000 33948       4       org.elasticsearch.common.util.concurrent.KeyedLock::acquire (96 bytes)
Event: 223703.480 Thread 0x00007fd474179800 nmethod 33947 0x00007fd4676b8ad0 code [0x00007fd4676b8ce0, 0x00007fd4676b9310]
Event: 223703.491 Thread 0x00007fd474173000 nmethod 33948 0x00007fd4690df490 code [0x00007fd4690df760, 0x00007fd4690e0888]
Event: 223766.418 Thread 0x00007fd474183800 33949   !   4       org.elasticsearch.index.engine.InternalEngine::get (469 bytes)
Event: 223766.448 Thread 0x00007fd474183800 nmethod 33949 0x00007fd46a1c1550 code [0x00007fd46a1c18e0, 0x00007fd46a1c3178]
Event: 223768.704 Thread 0x00007fd47418a800 33950       4       org.elasticsearch.action.get.GetResponse::writeTo (14 bytes)
Event: 223768.735 Thread 0x00007fd47418a800 nmethod 33950 0x00007fd46a1b8050 code [0x00007fd46a1b8540, 0x00007fd46a1ba5d0]
Event: 223983.366 Thread 0x00007fd474175000 33951       4       java.util.TimSort::mergeHi (732 bytes)
Event: 223983.388 Thread 0x00007fd474175000 nmethod 33951 0x00007fd46a1bf050 code [0x00007fd46a1bf240, 0x00007fd46a1c0818]
Event: 224018.553 Thread 0x00007fd474191000 33952   !   3       org.joda.time.format.DateTimeFormatterBuilder$NumberFormatter::parseInto (362 bytes)

GC Heap History (10 events):
Event: 224016.537 GC heap before
{Heap before GC invocations=174448 (full 69):
 par new generation   total 996800K, used 966413K [0x0000000280000000, 0x00000002c3990000, 0x00000002c3990000)
  eden space 886080K, 100% used [0x0000000280000000, 0x00000002b6150000, 0x00000002b6150000)
  from space 110720K,  72% used [0x00000002b6150000, 0x00000002bafc34c8, 0x00000002bcd70000)
  to   space 110720K,   0% used [0x00000002bcd70000, 0x00000002bcd70000, 0x00000002c3990000)
 concurrent mark-sweep generation total 20912576K, used 12689381K [0x00000002c3990000, 0x00000007c0000000, 0x00000007c0000000)
 Metaspace       used 91771K, capacity 99868K, committed 100984K, reserved 1136640K
  class space    used 11496K, capacity 13754K, committed 14048K, reserved 1048576K
Event: 224016.560 GC heap after
Heap after GC invocations=174449 (full 69):
 par new generation   total 996800K, used 73244K [0x0000000280000000, 0x00000002c3990000, 0x00000002c3990000)
  eden space 886080K,   0% used [0x0000000280000000, 0x0000000280000000, 0x00000002b6150000)
  from space 110720K,  66% used [0x00000002bcd70000, 0x00000002c14f7080, 0x00000002c3990000)
  to   space 110720K,   0% used [0x00000002b6150000, 0x00000002b6150000, 0x00000002bcd70000)
 concurrent mark-sweep generation total 20912576K, used 12693999K [0x00000002c3990000, 0x00000007c0000000, 0x00000007c0000000)
 Metaspace       used 91771K, capacity 99868K, committed 100984K, reserved 1136640K
  class space    used 11496K, capacity 13754K, committed 14048K, reserved 1048576K
}
Event: 224016.788 GC heap before
{Heap before GC invocations=174449 (full 69):
 par new generation   total 996800K, used 959324K [0x0000000280000000, 0x00000002c3990000, 0x00000002c3990000)
  eden space 886080K, 100% used [0x0000000280000000, 0x00000002b6150000, 0x00000002b6150000)
  from space 110720K,  66% used [0x00000002bcd70000, 0x00000002c14f7080, 0x00000002c3990000)
  to   space 110720K,   0% used [0x00000002b6150000, 0x00000002b6150000, 0x00000002bcd70000)
 concurrent mark-sweep generation total 20912576K, used 12693999K [0x00000002c3990000, 0x00000007c0000000, 0x00000007c0000000)
 Metaspace       used 91771K, capacity 99868K, committed 100984K, reserved 1136640K
  class space    used 11496K, capacity 13754K, committed 14048K, reserved 1048576K
Event: 224016.813 GC heap after
Heap after GC invocations=174450 (full 69):
 par new generation   total 996800K, used 75170K [0x0000000280000000, 0x00000002c3990000, 0x00000002c3990000)
  eden space 886080K,   0% used [0x0000000280000000, 0x0000000280000000, 0x00000002b6150000)
  from space 110720K,  67% used [0x00000002b6150000, 0x00000002baab8ad0, 0x00000002bcd70000)
  to   space 110720K,   0% used [0x00000002bcd70000, 0x00000002bcd70000, 0x00000002c3990000)
 concurrent mark-sweep generation total 20912576K, used 12699016K [0x00000002c3990000, 0x00000007c0000000, 0x00000007c0000000)
 Metaspace       used 91771K, capacity 99868K, committed 100984K, reserved 1136640K
  class space    used 11496K, capacity 13754K, committed 14048K, reserved 1048576K
}
Event: 224016.937 GC heap before
{Heap before GC invocations=174450 (full 69):
 par new generation   total 996800K, used 961250K [0x0000000280000000, 0x00000002c3990000, 0x00000002c3990000)
  eden space 886080K, 100% used [0x0000000280000000, 0x00000002b6150000, 0x00000002b6150000)
  from space 110720K,  67% used [0x00000002b6150000, 0x00000002baab8ad0, 0x00000002bcd70000)
  to   space 110720K,   0% used [0x00000002bcd70000, 0x00000002bcd70000, 0x00000002c3990000)
 concurrent mark-sweep generation total 20912576K, used 12699016K [0x00000002c3990000, 0x00000007c0000000, 0x00000007c0000000)
 Metaspace       used 91771K, capacity 99868K, committed 100984K, reserved 1136640K
  class space    used 11496K, capacity 13754K, committed 14048K, reserved 1048576K
Event: 224016.960 GC heap after
Heap after GC invocations=174451 (full 69):
 par new generation   total 996800K, used 70821K [0x0000000280000000, 0x00000002c3990000, 0x00000002c3990000)
  eden space 886080K,   0% used [0x0000000280000000, 0x0000000280000000, 0x00000002b6150000)
  from space 110720K,  63% used [0x00000002bcd70000, 0x00000002c1299670, 0x00000002c3990000)
  to   space 110720K,   0% used [0x00000002b6150000, 0x00000002b6150000, 0x00000002bcd70000)
 concurrent mark-sweep generation total 20912576K, used 12705741K [0x00000002c3990000, 0x00000007c0000000, 0x00000007c0000000)
 Metaspace       used 91771K, capacity 99868K, committed 100984K, reserved 1136640K
  class space    used 11496K, capacity 13754K, committed 14048K, reserved 1048576K
}
Event: 224017.195 GC heap before
{Heap before GC invocations=174451 (full 69):
 par new generation   total 996800K, used 956901K [0x0000000280000000, 0x00000002c3990000, 0x00000002c3990000)
  eden space 886080K, 100% used [0x0000000280000000, 0x00000002b6150000, 0x00000002b6150000)
  from space 110720K,  63% used [0x00000002bcd70000, 0x00000002c1299670, 0x00000002c3990000)
  to   space 110720K,   0% used [0x00000002b6150000, 0x00000002b6150000, 0x00000002bcd70000)
 concurrent mark-sweep generation total 20912576K, used 12705741K [0x00000002c3990000, 0x00000007c0000000, 0x00000007c0000000)
 Metaspace       used 91771K, capacity 99868K, committed 100984K, reserved 1136640K
  class space    used 11496K, capacity 13754K, committed 14048K, reserved 1048576K
Event: 224017.221 GC heap after
Heap after GC invocations=174452 (full 69):
 par new generation   total 996800K, used 82581K [0x0000000280000000, 0x00000002c3990000, 0x00000002c3990000)
  eden space 886080K,   0% used [0x0000000280000000, 0x0000000280000000, 0x00000002b6150000)
  from space 110720K,  74% used [0x00000002b6150000, 0x00000002bb1f5660, 0x00000002bcd70000)
  to   space 110720K,   0% used [0x00000002bcd70000, 0x00000002bcd70000, 0x00000002c3990000)
 concurrent mark-sweep generation total 20912576K, used 12714603K [0x00000002c3990000, 0x00000007c0000000, 0x00000007c0000000)
 Metaspace       used 91771K, capacity 99868K, committed 100984K, reserved 1136640K
  class space    used 11496K, capacity 13754K, committed 14048K, reserved 1048576K
}
Event: 224018.526 GC heap before
{Heap before GC invocations=174452 (full 69):
 par new generation   total 996800K, used 968661K [0x0000000280000000, 0x00000002c3990000, 0x00000002c3990000)
  eden space 886080K, 100% used [0x0000000280000000, 0x00000002b6150000, 0x00000002b6150000)
  from space 110720K,  74% used [0x00000002b6150000, 0x00000002bb1f5660, 0x00000002bcd70000)
  to   space 110720K,   0% used [0x00000002bcd70000, 0x00000002bcd70000, 0x00000002c3990000)
 concurrent mark-sweep generation total 20912576K, used 12714603K [0x00000002c3990000, 0x00000007c0000000, 0x00000007c0000000)
 Metaspace       used 91771K, capacity 99868K, committed 100984K, reserved 1136640K
  class space    used 11496K, capacity 13754K, committed 14048K, reserved 1048576K
Event: 224018.549 GC heap after
Heap after GC invocations=174453 (full 69):
 par new generation   total 996800K, used 80247K [0x0000000280000000, 0x00000002c3990000, 0x00000002c3990000)
  eden space 886080K,   0% used [0x0000000280000000, 0x0000000280000000, 0x00000002b6150000)
  from space 110720K,  72% used [0x00000002bcd70000, 0x00000002c1bcdde0, 0x00000002c3990000)
  to   space 110720K,   0% used [0x00000002b6150000, 0x00000002b6150000, 0x00000002bcd70000)
 concurrent mark-sweep generation total 20912576K, used 12725547K [0x00000002c3990000, 0x00000007c0000000, 0x00000007c0000000)
 Metaspace       used 91771K, capacity 99868K, committed 100984K, reserved 1136640K
  class space    used 11496K, capacity 13754K, committed 14048K, reserved 1048576K
}

Deoptimization events (10 events):
Event: 223698.613 Thread 0x00007fd304014800 Uncommon trap: reason=unstable_if action=reinterpret pc=0x00007fd4678242b8 method=org.elasticsearch.common.util.concurrent.KeyedLock.tryCreateNewLock(Ljava/lang/Object;)Lorg/elasticsearch/common/util/concurrent/KeyedLock$ReleasableLock; @ 32
Event: 224018.550 Thread 0x00007fd2d4015000 Uncommon trap: reason=range_check action=none pc=0x00007fd469a0ad24 method=java.math.BigDecimal.<init>([CIILjava/math/MathContext;)V @ 43
Event: 224018.550 Thread 0x00007fd2dc014800 Uncommon trap: reason=range_check action=none pc=0x00007fd469a0ad24 method=java.math.BigDecimal.<init>([CIILjava/math/MathContext;)V @ 43
Event: 224018.550 Thread 0x00007fd2dc014800 Uncommon trap: reason=range_check action=none pc=0x00007fd469a0ad24 method=java.math.BigDecimal.<init>([CIILjava/math/MathContext;)V @ 43
Event: 224018.550 Thread 0x00007fd2d4015000 Uncommon trap: reason=range_check action=none pc=0x00007fd469a0ad24 method=java.math.BigDecimal.<init>([CIILjava/math/MathContext;)V @ 43
Event: 224018.551 Thread 0x00007fd2d4015000 Uncommon trap: reason=unstable_if action=reinterpret pc=0x00007fd46a00fc78 method=org.elasticsearch.index.search.QueryParserHelper.resolveMappingField(Lorg/elasticsearch/index/query/QueryShardContext;Ljava/lang/String;FZZLjava/lang/String;)Ljava/util/Map;
Event: 224018.552 Thread 0x00007fd2d4015000 Uncommon trap: reason=unstable_if action=reinterpret pc=0x00007fd46a168530 method=org.joda.time.format.DateTimeFormatterBuilder$NumberFormatter.parseInto(Lorg/joda/time/format/DateTimeParserBucket;Ljava/lang/CharSequence;I)I @ 105
Event: 224018.552 Thread 0x00007fd2d4015000 Uncommon trap: reason=unstable_if action=reinterpret pc=0x00007fd469a0ad8c method=java.math.BigDecimal.<init>([CIILjava/math/MathContext;)V @ 396
Event: 224018.553 Thread 0x00007fd2d4015000 Uncommon trap: reason=unstable_if action=reinterpret pc=0x00007fd4676ac2d0 method=org.apache.lucene.queryparser.classic.QueryParserTokenManager.jjMoveNfa_2(II)I @ 416
Event: 224018.553 Thread 0x00007fd2d4015000 Uncommon trap: reason=unstable_if action=reinterpret pc=0x00007fd469e16ba0 method=org.apache.lucene.queryparser.classic.QueryParserTokenManager.getNextToken()Lorg/apache/lucene/queryparser/classic/Token; @ 120

Internal exceptions (10 events):
Event: 223993.204 Thread 0x00007fd360005800 Exception <a 'sun/nio/fs/UnixException'> (0x000000028e7505a8) thrown at [/HUDSON/workspace/8-2-build-linux-amd64/jdk8u151/9699/hotspot/src/share/vm/prims/jni.cpp, line 709]
Event: 223993.205 Thread 0x00007fd360005800 Exception <a 'sun/nio/fs/UnixException'> (0x000000028e7512f0) thrown at [/HUDSON/workspace/8-2-build-linux-amd64/jdk8u151/9699/hotspot/src/share/vm/prims/jni.cpp, line 709]
Event: 223993.205 Thread 0x00007fd360005800 Exception <a 'sun/nio/fs/UnixException'> (0x000000028e752028) thrown at [/HUDSON/workspace/8-2-build-linux-amd64/jdk8u151/9699/hotspot/src/share/vm/prims/jni.cpp, line 709]
Event: 223993.205 Thread 0x00007fd360005800 Exception <a 'sun/nio/fs/UnixException'> (0x000000028e752d48) thrown at [/HUDSON/workspace/8-2-build-linux-amd64/jdk8u151/9699/hotspot/src/share/vm/prims/jni.cpp, line 709]
Event: 223993.205 Thread 0x00007fd360005800 Exception <a 'sun/nio/fs/UnixException'> (0x000000028e753a88) thrown at [/HUDSON/workspace/8-2-build-linux-amd64/jdk8u151/9699/hotspot/src/share/vm/prims/jni.cpp, line 709]
Event: 223993.205 Thread 0x00007fd360005800 Exception <a 'sun/nio/fs/UnixException'> (0x000000028e7547c0) thrown at [/HUDSON/workspace/8-2-build-linux-amd64/jdk8u151/9699/hotspot/src/share/vm/prims/jni.cpp, line 709]
Event: 224018.550 Thread 0x00007fd2dc014800 Exception <a 'java/lang/ArrayIndexOutOfBoundsException': 0> (0x00000002804c55c8) thrown at [/HUDSON/workspace/8-2-build-linux-amd64/jdk8u151/9699/hotspot/src/share/vm/interpreter/interpreterRuntime.cpp, line 366]
Event: 224018.550 Thread 0x00007fd2d4015000 Exception <a 'java/lang/ArrayIndexOutOfBoundsException': 0> (0x00000002804c4b50) thrown at [/HUDSON/workspace/8-2-build-linux-amd64/jdk8u151/9699/hotspot/src/share/vm/interpreter/interpreterRuntime.cpp, line 366]
Event: 224018.550 Thread 0x00007fd2dc014800 Exception <a 'java/lang/ArrayIndexOutOfBoundsException': 0> (0x00000002806d4598) thrown at [/HUDSON/workspace/8-2-build-linux-amd64/jdk8u151/9699/hotspot/src/share/vm/interpreter/interpreterRuntime.cpp, line 366]
Event: 224018.550 Thread 0x00007fd2d4015000 Exception <a 'java/lang/ArrayIndexOutOfBoundsException': 0> (0x00000002806d3e98) thrown at [/HUDSON/workspace/8-2-build-linux-amd64/jdk8u151/9699/hotspot/src/share/vm/interpreter/interpreterRuntime.cpp, line 366]

Events (10 events):
Event: 224018.552 Thread 0x00007fd2d4015000 DEOPT UNPACKING pc=0x00007fd465004f69 sp=0x00007f7b21e0feb8 mode 2
Event: 224018.552 Thread 0x00007fd2d4015000 Uncommon trap: trap_request=0xffffff65 fr.pc=0x00007fd469a0ad8c
Event: 224018.552 Thread 0x00007fd2d4015000 DEOPT PACKING pc=0x00007fd469a0ad8c sp=0x00007f7b21e0ff30
Event: 224018.552 Thread 0x00007fd2d4015000 DEOPT UNPACKING pc=0x00007fd465004f69 sp=0x00007f7b21e0fed8 mode 2
Event: 224018.553 Thread 0x00007fd2d4015000 Uncommon trap: trap_request=0xffffff65 fr.pc=0x00007fd4676ac2d0
Event: 224018.553 Thread 0x00007fd2d4015000 DEOPT PACKING pc=0x00007fd4676ac2d0 sp=0x00007f7b21e10980
Event: 224018.553 Thread 0x00007fd2d4015000 DEOPT UNPACKING pc=0x00007fd465004f69 sp=0x00007f7b21e10938 mode 2
Event: 224018.553 Thread 0x00007fd2d4015000 Uncommon trap: trap_request=0xffffff65 fr.pc=0x00007fd469e16ba0
Event: 224018.553 Thread 0x00007fd2d4015000 DEOPT PACKING pc=0x00007fd469e16ba0 sp=0x00007f7b21e10a10
Event: 224018.553 Thread 0x00007fd2d4015000 DEOPT UNPACKING pc=0x00007fd465004f69 sp=0x00007f7b21e109d0 mode 2


Dynamic libraries:
00400000-00401000 r-xp 00000000 fd:01 1310842                            /usr/local/java/jdk1.8.0_151/bin/java
00600000-00601000 rw-p 00000000 fd:01 1310842                            /usr/local/java/jdk1.8.0_151/bin/java
022a2000-022c3000 rw-p 00000000 00:00 0                                  [heap]
022c3000-022e4000 rw-p 00000000 00:00 0                                  [heap]
280000000-7c0240000 rw-p 00000000 00:00 0 
7c0240000-7c0db8000 rw-p 00000000 00:00 0 
7c0db8000-800000000 ---p 00000000 00:00 0 
7f565f322000-7f5673d9b000 r--s 00000000 fd:11 117310850                  /usr/local/elasticsearch-6.6.0/data/nodes/0/indices/Alz-AshMRqWBq0IwafhiAg/1/index/_ug7_Lucene70_0.dvd
7f5673d9b000-7f567b309000 r--s 40000000 fd:11 117309679                  /usr/local/elasticsearch-6.6.0/data/nodes/0/indices/Alz-AshMRqWBq0IwafhiAg/1/index/_ug7.fdt
7f567b309000-7f56bb309000 r--s 00000000 fd:11 117309679                  /usr/local/elasticsearch-6.6.0/data/nodes/0/indices/Alz-AshMRqWBq0IwafhiAg/1/index/_ug7.fdt
7f56bb309000-7f56d6f95000 r--s 00000000 fd:11 117310581                  /usr/local/elasticsearch-6.6.0/data/nodes/0/indices/Alz-AshMRqWBq0IwafhiAg/1/index/_ug7_Lucene50_0.tim
7f56d6f95000-7f56e79c6000 r--s 00000000 fd:11 117310564                  /usr/local/elasticsearch-6.6.0/data/nodes/0/indices/Alz-AshMRqWBq0IwafhiAg/1/index/_ug7_Lucene50_0.pos
7f56e79c6000-7f5700555000 r--s 00000000 fd:11 117310548                  /usr/local/elasticsearch-6.6.0/data/nodes/0/indices/Alz-AshMRqWBq0IwafhiAg/1/index/_ug7_Lucene50_0.doc

...

复制