Announcement Announcement Module
No announcement yet.
Debug connections to mysql from spring application Page Title Module
Move Remove Collapse
Conversation Detail Module
  • Filter
  • Time
  • Show
Clear All
new posts

  • Debug connections to mysql from spring application


    Our Spring application has jackrabbit content repository. It runs quartz based crons. It uses hibernate underneath.

    We are seeing the following exception in the logs.
    2013-06-05 12:12:27,616 [ClusterNode-node1] ERROR org.apache.jackrabbit.core.cluster.ClusterNode - Periodic sync of journal failed: Couldn't sync the cluster node
    org.apache.jackrabbit.core.cluster.ClusterExceptio n: Couldn't sync the cluster node
    at org.apache.jackrabbit.core.cluster.ClusterNode.syn c(
    at (
    Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsExce ption: Communications link failure

    The last packet successfully received from the server was 362,492 milliseconds ago. The last packet sent successfully to the server was 19,111 milliseconds ago.
    at sun.reflect.NativeConstructorAccessorImpl.newInsta nce0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInsta nce(
    at sun.reflect.DelegatingConstructorAccessorImpl.newI nstance(
    at java.lang.reflect.Constructor.newInstance(Construc
    at com.mysql.jdbc.Util.handleNewInstance( 1)
    at com.mysql.jdbc.SQLError.createCommunicationsExcept ion(
    at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO. java:3102)
    at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO. java:2991)
    at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.ja va:3532)
    at com.mysql.jdbc.MysqlIO.sendCommand( 02)
    at com.mysql.jdbc.MysqlIO.sqlQueryDirect( :2163)
    at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionIm
    at com.mysql.jdbc.ConnectionImpl.setAutoCommit(Connec
    at org.apache.commons.dbcp.DelegatingConnection.setAu toCommit(
    at org.apache.commons.dbcp.DelegatingConnection.setAu toCommit(
    at org.apache.commons.dbcp.PoolingDataSource$PoolGuar dConnectionWrapper.setAutoCommit(PoolingDataSource .java:328)
    at org.apache.jackrabbit.core.util.db.ConnectionHelpe r.startBatch(
    at org.apache.jackrabbit.core.journal.DatabaseJournal .startBatch(
    at org.apache.jackrabbit.core.journal.DatabaseJournal .doSync(
    at org.apache.jackrabbit.core.journal.AbstractJournal .internalSync(
    at org.apache.jackrabbit.core.journal.AbstractJournal .sync(
    at org.apache.jackrabbit.core.cluster.ClusterNode.syn c(
    ... 2 more
    Caused by: Can not read response from server. Expected to read 4 bytes, read 0 bytes before connection was unexpectedly lost.
    at com.mysql.jdbc.MysqlIO.readFully( )
    at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO. java:3002)
    ... 17 more
    2013-06-05 12:12:29,934 [QuartzScheduler_org.springframework.scheduling.qua dler] ERROR org.springframework.scheduling.quartz.LocalDataSou rceJobStore - MisfireHandler: Error handling misfires: Failed to obtain DB connection from data source ' ing.quartz.SchedulerFactoryBean#0': java.sql.SQLException: java.lang.OutOfMemoryError: Java heap space
    org.quartz.JobPersistenceException: Failed to obtain DB connection from data source ' ing.quartz.SchedulerFactoryBean#0': java.sql.SQLException: java.lang.OutOfMemoryError: Java heap space [See nested exception: java.sql.SQLException: java.lang.OutOfMemoryError: Java heap space]
    at org.quartz.impl.jdbcjobstore.JobStoreCMT.getNonMan agedTXConnection(
    at org.quartz.impl.jdbcjobstore.JobStoreSupport.doRec overMisfires(
    at org.quartz.impl.jdbcjobstore.JobStoreSupport$Misfi reHandler.manage(
    at org.quartz.impl.jdbcjobstore.JobStoreSupport$Misfi
    Caused by: java.sql.SQLException: java.lang.OutOfMemoryError: Java heap space

    We are seeing that our spring application has around 73 connections to our remote mysql server.
    netstat -a | grep mysql | wc -l

    1. How do I find which class in the application corresponds to each of the 63 connections.
    This number starts with around 14 and increases to a large number like 73. The cause needs to be investigated.
    2. How do I find the DB username used for each of these connections.
    3. I had seen around 87 threads running under tomcat at the time of the error. Please find them attached in this email
    4. Some pointers on how to proceed further would help.