Announcement Announcement Module
Collapse
No announcement yet.
Debug connections to mysql from spring application Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • Debug connections to mysql from spring application

    Hi,

    Our Spring application has jackrabbit content repository. It runs quartz based crons. It uses hibernate underneath.

    We are seeing the following exception in the logs.
    2013-06-05 12:12:27,616 [ClusterNode-node1] ERROR org.apache.jackrabbit.core.cluster.ClusterNode - Periodic sync of journal failed: Couldn't sync the cluster node
    org.apache.jackrabbit.core.cluster.ClusterExceptio n: Couldn't sync the cluster node
    at org.apache.jackrabbit.core.cluster.ClusterNode.syn c(ClusterNode.java:331)
    at org.apache.jackrabbit.core.cluster.ClusterNode.run (ClusterNode.java:293)
    at java.lang.Thread.run(Thread.java:662)
    Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsExce ption: Communications link failure

    The last packet successfully received from the server was 362,492 milliseconds ago. The last packet sent successfully to the server was 19,111 milliseconds ago.
    at sun.reflect.NativeConstructorAccessorImpl.newInsta nce0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInsta nce(NativeConstructorAccessorImpl.java:39)
    at sun.reflect.DelegatingConstructorAccessorImpl.newI nstance(DelegatingConstructorAccessorImpl.java:27)
    at java.lang.reflect.Constructor.newInstance(Construc tor.java:513)
    at com.mysql.jdbc.Util.handleNewInstance(Util.java:41 1)
    at com.mysql.jdbc.SQLError.createCommunicationsExcept ion(SQLError.java:1116)
    at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO. java:3102)
    at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO. java:2991)
    at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.ja va:3532)
    at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:20 02)
    at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java :2163)
    at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionIm pl.java:2618)
    at com.mysql.jdbc.ConnectionImpl.setAutoCommit(Connec tionImpl.java:5000)
    at org.apache.commons.dbcp.DelegatingConnection.setAu toCommit(DelegatingConnection.java:371)
    at org.apache.commons.dbcp.DelegatingConnection.setAu toCommit(DelegatingConnection.java:371)
    at org.apache.commons.dbcp.PoolingDataSource$PoolGuar dConnectionWrapper.setAutoCommit(PoolingDataSource .java:328)
    at org.apache.jackrabbit.core.util.db.ConnectionHelpe r.startBatch(ConnectionHelper.java:240)
    at org.apache.jackrabbit.core.journal.DatabaseJournal .startBatch(DatabaseJournal.java:506)
    at org.apache.jackrabbit.core.journal.DatabaseJournal .doSync(DatabaseJournal.java:450)
    at org.apache.jackrabbit.core.journal.AbstractJournal .internalSync(AbstractJournal.java:206)
    at org.apache.jackrabbit.core.journal.AbstractJournal .sync(AbstractJournal.java:189)
    at org.apache.jackrabbit.core.cluster.ClusterNode.syn c(ClusterNode.java:328)
    ... 2 more
    Caused by: java.io.EOFException: Can not read response from server. Expected to read 4 bytes, read 0 bytes before connection was unexpectedly lost.
    at com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:2552 )
    at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO. java:3002)
    ... 17 more
    2013-06-05 12:12:29,934 [QuartzScheduler_org.springframework.scheduling.qua rtz.SchedulerFactoryBean#0-bf1-rmaicic1.ads.bf1.yahoo.com1370389426455_MisfireHan dler] ERROR org.springframework.scheduling.quartz.LocalDataSou rceJobStore - MisfireHandler: Error handling misfires: Failed to obtain DB connection from data source 'springNonTxDataSource.org.springframework.schedul ing.quartz.SchedulerFactoryBean#0': java.sql.SQLException: java.lang.OutOfMemoryError: Java heap space
    org.quartz.JobPersistenceException: Failed to obtain DB connection from data source 'springNonTxDataSource.org.springframework.schedul ing.quartz.SchedulerFactoryBean#0': java.sql.SQLException: java.lang.OutOfMemoryError: Java heap space [See nested exception: java.sql.SQLException: java.lang.OutOfMemoryError: Java heap space]
    at org.quartz.impl.jdbcjobstore.JobStoreCMT.getNonMan agedTXConnection(JobStoreCMT.java:165)
    at org.quartz.impl.jdbcjobstore.JobStoreSupport.doRec overMisfires(JobStoreSupport.java:3132)
    at org.quartz.impl.jdbcjobstore.JobStoreSupport$Misfi reHandler.manage(JobStoreSupport.java:3923)
    at org.quartz.impl.jdbcjobstore.JobStoreSupport$Misfi reHandler.run(JobStoreSupport.java:3943)
    Caused by: java.sql.SQLException: java.lang.OutOfMemoryError: Java heap space


    We are seeing that our spring application has around 73 connections to our remote mysql server.
    netstat -a | grep mysql | wc -l
    63


    1. How do I find which class in the application corresponds to each of the 63 connections.
    This number starts with around 14 and increases to a large number like 73. The cause needs to be investigated.
    2. How do I find the DB username used for each of these connections.
    3. I had seen around 87 threads running under tomcat at the time of the error. Please find them attached in this email
    4. Some pointers on how to proceed further would help.


    Thanks
    -Priyanka
Working...
X