线上一个IM机器 不停fullgc CPU load 上升,HEAP space 100% used
环境信息
java version "1.6.0_27-ea"
Java(TM) SE Runtime Environment (build 1.6.0_27-ea-b03)
Java HotSpot(TM) 64-Bit Server VM (build 20.2-b03, mixed mode)
问题定位
jstack -l $(pgrep java)
半天不响应,换-F模式
jstack -F $(pgrep java)
控制台报NPE
(NullPointException),所有dump后的线程状态全部为BLOCKED:
Deadlock Detection:
Can't print deadlocks:null
Thread 6125: (state = BLOCKED)
- java.lang.Object) @bci=0, line=133 (Compiled frame)
- int, boolean) @bci=20, line=294 (Compiled frame)
- int) @bci=4, line=324 (Compiled frame)
- int) @bci=3, line=970 (Compiled frame)
- @bci=236, line=911 (Compiled frame)
- @bci=1, line=836 (Compiled frame)
- @bci=134, line=1014 (Compiled frame)
- com.caucho.hessian.client.HessianURLConnection.getOutputStream() @bci=4, line=97 (Compiled frame)
- com.caucho.hessian.client.HessianProxy.sendRequest(java.lang.String, java.lang.Object[]) @bci=31, line=279 (Compiled frame)
- com.caucho.hessian.client.HessianProxy.invoke(java.lang.Object, java.lang.reflect.Method, java.lang.Object[]) @bci=388, line=171 (Compiled frame)
- $Proxy5.getTokenByUid(java.lang.String) @bci=16 (Compiled frame)
- com.uplus.im.websock.session.WebSockSession.onTextMessage(java.nio.CharBuffer) @bci=64, line=66 (Compiled frame)
- org.apache.catalina.websocket.MessageInbound.onTextData(java.io.Reader) @bci=80, line=74 (Compiled frame)
- org.apache.catalina.websocket.StreamInbound.doOnTextData(java.io.Reader) @bci=19, line=187 (Compiled frame)
- org.apache.catalina.websocket.StreamInbound.onData() @bci=88, line=135 (Compiled frame)
- org.apache.coyote.http11.upgrade.UpgradeProcessor.upgradeDispatch() @bci=4, line=83 (Interpreted frame)
- @bci=146, line=587 (Interpreted frame)
- @bci=129, line=1686 (Compiled frame)
- java.util.concurrent.ThreadPoolExecutor$Worker.runTask(java.lang.Runnable) @bci=59, line=886 (Compiled frame)
- java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=28, line=908 (Compiled frame)
- java.lang.Thread.run() @bci=11, line=662 (Interpreted frame)
Thread 6124: (state = BLOCKED)
- java.lang.Object) @bci=0, line=133 (Compiled frame)
- int, boolean) @bci=20, line=294 (Compiled frame)
- int) @bci=4, line=324 (Compiled frame)
- int) @bci=3, line=970 (Compiled frame)
- @bci=236, line=911 (Compiled frame)
- @bci=1, line=836 (Compiled frame)
- @bci=134, line=1014 (Compiled frame)
- com.caucho.hessian.client.HessianURLConnection.getOutputStream() @bci=4, line=97 (Compiled frame)
- com.caucho.hessian.client.HessianProxy.sendRequest(java.lang.String, java.lang.Object[]) @bci=31, line=279 (Compiled frame)
- com.caucho.hessian.client.HessianProxy.invoke(java.lang.Object, java.lang.reflect.Method, java.lang.Object[]) @bci=388, line=171 (Compiled frame)
- $Proxy5.getTokenByUid(java.lang.String) @bci=16 (Compiled frame)
- com.uplus.im.websock.session.WebSockSession.onTextMessage(java.nio.CharBuffer) @bci=64, line=66 (Compiled frame)
- org.apache.catalina.websocket.MessageInbound.onTextData(java.io.Reader) @bci=80, line=74 (Compiled frame)
- org.apache.catalina.websocket.StreamInbound.doOnTextData(java.io.Reader) @bci=19, line=187 (Compiled frame)
- org.apache.catalina.websocket.StreamInbound.onData() @bci=88, line=135 (Compiled frame)
- org.apache.coyote.http11.upgrade.UpgradeProcessor.upgradeDispatch() @bci=4, line=83 (Interpreted frame)
- @bci=146, line=587 (Interpreted frame)
- @bci=129, line=1686 (Compiled frame)
- java.util.concurrent.ThreadPoolExecutor$Worker.runTask(java.lang.Runnable) @bci=59, line=886 (Compiled frame)
- java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=28, line=908 (Compiled frame)
- java.lang.Thread.run() @bci=11, line=662 (Interpreted frame)