Announcement Announcement Module
Collapse
No announcement yet.
JobRestartException is being thrown in another thread Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • JobRestartException is being thrown in another thread

    Dear Spring Batch developers,

    In my case the previous job execution has failed with the following trace:

    Code:
    2011-07-07 05:14:11,364 ERROR [org.springframework.batch.core.job.AbstractJob] Encountered interruption executing job
    org.springframework.batch.core.JobInterruptedException: Step requested termination: StepExecution: id=11, version=30, name=documentImageIndexerStep, status=UNKNOWN, exitStatus=FAILED, readCount=29, filterCount=3, writeCount=25 readSkipCount=0, writeSkipCount=0, processSkipCount=0, commitCount=28, rollbackCount=2, exitDescription=org.springframework.transaction.TransactionSystemException: Could not roll back Hibernate transaction; nested exception is org.hibernate.TransactionException: JDBC rollback failed
            at org.springframework.orm.hibernate3.HibernateTransactionManager.doRollback(HibernateTransactionManager.java:679)
            at org.springframework.transaction.support.AbstractPlatformTransactionManager.processRollback(AbstractPlatformTransactionManager.java:845)
            at org.springframework.transaction.support.AbstractPlatformTransactionManager.rollback(AbstractPlatformTransactionManager.java:822)
            at org.springframework.transaction.support.TransactionTemplate.rollbackOnException(TransactionTemplate.java:161)
            at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:134)
    Caused by: org.hibernate.TransactionException: JDBC rollback failed
            at org.hibernate.transaction.JDBCTransaction.rollback(JDBCTransaction.java:200)
            at org.springframework.orm.hibernate3.HibernateTransactionManager.doRollback(HibernateTransactionManager.java:676)
            ... 22 more
    Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: Communications link failure during rollback(). Transaction resolution unknown.
    so Spring batch was not able to update the execution status. When the job is attempted to be restarted, the following exception is raised:

    Code:
    2011-07-07 10:32:30,750 ERROR [org.springframework.batch.core.job.AbstractJob] Encountered fatal error executing job
    org.springframework.batch.core.repository.JobRestartException: Cannot restart step from UNKNOWN status. The last execution ended with a failure that could not be rolled back, so it may be dangerous to proceed. Manual intervention is probably necessary.
            at org.springframework.batch.core.job.SimpleStepHandler.shouldStart(SimpleStepHandler.java:197)
            at org.springframework.batch.core.job.SimpleStepHandler.handleStep(SimpleStepHandler.java:117)
            at org.springframework.batch.core.job.flow.JobFlowExecutor.executeStep(JobFlowExecutor.java:61)
            at org.springframework.batch.core.job.flow.support.state.StepState.handle(StepState.java:60)
            at org.springframework.batch.core.job.flow.support.SimpleFlow.resume(SimpleFlow.java:144)
            at org.springframework.batch.core.job.flow.support.SimpleFlow.start(SimpleFlow.java:124)
            at org.springframework.batch.core.job.flow.FlowJob.doExecute(FlowJob.java:135)
            at org.springframework.batch.core.job.AbstractJob.execute(AbstractJob.java:281)
            at org.springframework.batch.core.launch.support.SimpleJobLauncher$1.run(SimpleJobLauncher.java:120)
            at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
            at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
            at java.lang.Thread.run(Thread.java:662)
    The problem is that this exception is raised in a new thread, and it is difficult to handle it. The "manual intervention" means that somebody needs to check logs and update the record in DB. This is too difficult ask for somebody who maintains the production environment. I would like the software to be able to react in fully automated manner.

    My suggestion is: Is is possible to check this pre-condition in advance and to raise the exception via JobLauncher#run(), so it is thrown in the thread that launches the job? What is happening now: the launcher reports that the job has started the execution, but very soon it fails...

    Any advise is welcomed.
    Last edited by dma_k; Jul 9th, 2011, 02:38 PM.

  • #2
    If the JobExecution has status UNKNOWN, I guess the JobLauncher could report that before trying to start a new one (feel free to raise a ticket in JIRA https://jira.springsource.org/browse/BATCH). It would still be in a background thread in your case, so how would it help exactly?

    Comment


    • #3
      Dave, now it is reported like you've described: the exception is raised after the job was launched in separate thread. Perhaps you can shift checking of this pre-condition from SimpleStepHandler to SimpleJobLauncher.run(). I have raised BATCH-1770.

      Comment

      Working...
      X