性能问答>centos服务器CPU被GC task占满,程序无内存溢出>
1回复

centos服务器CPU被GC task占满,程序无内存溢出



环境:服务器为centos,2个8核16线CPU,内存64G,jdk版本1.7.0_67

问题描述:服务器部署的java应用,jvm堆内存为8G,启动一段时间后CPU占用飙升到2300%

jstack查看获得如下信息:

"VM Thread" prio=10 tid=0x00007f5ac4095800 nid=0xa599 runnable

"GC task thread#0 (ParallelGC)" prio=10 tid=0x00007f5ac4021000 nid=0xa56b runnable
"GC task thread#1 (ParallelGC)" prio=10 tid=0x00007f5ac4023000 nid=0xa56c runnable
"GC task thread#2 (ParallelGC)" prio=10 tid=0x00007f5ac4025000 nid=0xa56d runnable
"GC task thread#3 (ParallelGC)" prio=10 tid=0x00007f5ac4026800 nid=0xa56e runnable
"GC task thread#4 (ParallelGC)" prio=10 tid=0x00007f5ac4028800 nid=0xa56f runnable
"GC task thread#5 (ParallelGC)" prio=10 tid=0x00007f5ac402a800 nid=0xa570 runnable
"GC task thread#6 (ParallelGC)" prio=10 tid=0x00007f5ac402c800 nid=0xa571 runnable
"GC task thread#7 (ParallelGC)" prio=10 tid=0x00007f5ac402e800 nid=0xa572 runnable
"GC task thread#8 (ParallelGC)" prio=10 tid=0x00007f5ac4030800 nid=0xa573 runnable
"GC task thread#9 (ParallelGC)" prio=10 tid=0x00007f5ac4032800 nid=0xa574 runnable
"GC task thread#10 (ParallelGC)" prio=10 tid=0x00007f5ac4034000 nid=0xa575 runnable
"GC task thread#11 (ParallelGC)" prio=10 tid=0x00007f5ac4036000 nid=0xa576 runnable
"GC task thread#12 (ParallelGC)" prio=10 tid=0x00007f5ac4038000 nid=0xa577 runnable
"GC task thread#13 (ParallelGC)" prio=10 tid=0x00007f5ac403a000 nid=0xa578 runnable
"GC task thread#14 (ParallelGC)" prio=10 tid=0x00007f5ac403c000 nid=0xa579 runnable
"GC task thread#15 (ParallelGC)" prio=10 tid=0x00007f5ac403e000 nid=0xa57a runnable
"GC task thread#16 (ParallelGC)" prio=10 tid=0x00007f5ac403f800 nid=0xa57b runnable
"GC task thread#17 (ParallelGC)" prio=10 tid=0x00007f5ac4041800 nid=0xa57c runnable
"GC task thread#18 (ParallelGC)" prio=10 tid=0x00007f5ac4043800 nid=0xa57d runnable
"GC task thread#19 (ParallelGC)" prio=10 tid=0x00007f5ac4045800 nid=0xa57e runnable
"GC task thread#20 (ParallelGC)" prio=10 tid=0x00007f5ac4047800 nid=0xa57f runnable
"GC task thread#21 (ParallelGC)" prio=10 tid=0x00007f5ac4049800 nid=0xa580 runnable
"GC task thread#22 (ParallelGC)" prio=10 tid=0x00007f5ac404b800 nid=0xa581 runnable
"VM Periodic Task Thread" prio=10 tid=0x00007f5ac40d5800 nid=0xa5a3 waiting on condition

JNI global references: 760
heap信息如下:

 PSYoungGen      
total 5208064K, used 1186489K [0x00000006b1000000, 0x0000000800000000, 0x0000000800000000)
eden space 4918272K, 20% used [0x00000006b1000000,0x00000006ef010b20,0x00000007dd300000)
from space 289792K, 58% used [0x00000007ee500000,0x00000007f8b9db50,0x0000000800000000)
to   space 280576K, 0% used [0x00000007dd300000,0x00000007dd300000,0x00000007ee500000)
 ParOldGen       
total 1714688K, used 705292K [0x0000000413000000, 0x000000047ba80000, 0x00000006b1000000)
object space 1714688K, 41% used [0x0000000413000000,0x000000043e0c3378,0x000000047ba80000)

dump下来6G多的内存快照用MAT做分析,疑似溢出只有38Mb,看了一下是设计的阻塞队列,基本可以排除内存溢出

出问题部分的代码并行2个线程,一个处理json的序列化与反序列化以及zip流的写出操作,json用的是jackson2.3,zip流方面检查过了没有资源未关闭导致溢出的情况;还有一个对处理完的数据进行推送,收集到指定数量或等待指定时间才做一次推送

现在能想到可能的情况都做了排查,本机单独运行出问题的模块也没什么问题,有没有遇到过类似问题的朋友,还望不吝赐教…

3981 阅读
请先登录,查看1条精彩评论吧
快去登录吧,你将获得
  • 浏览更多精彩评论
  • 和开发者讨论交流,共同进步