Announcement Announcement Module
Collapse
No announcement yet.
Spring Batch Restart problem Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • Spring Batch Restart problem

    HI, I have 3 sql loader steps in my job and I check for errors in the sql loader output logfile before I move from first sql loader step to second sql loader step. This checking is done through decision tag using a decider. The scenario is job fails at insertIntoItemMasterStep which means 3 sql loader steps are completed and the decision is "COMPLETED" after every sql loader step. When I restart the job, the job tries to execute "decision1" which is the decider after first sql load. Ideally it shouldn't bother about any of the succeeded steps and should execute only from insertIntoItemMasterStep. Why does the flow goes back to decision1? The decision1 fails because it expects some runtime value that should be passed via first sql loader step. Please find below the stacktrace and also the job config. Any help in this regard is appreciated. I tried to narrow down the problem my removing all decision tags and was able to restart the job from insertIntoItemMasterStep.

    Code:
    2010-06-28 15:44:30,064 DEBUG BatchJobRunner  - Job Name = itemMasterDailyUploadJob, Job Order Id=222, Job Context = itemMasterDailyUploadJob-context.xml
    starting job with job context =itemMasterDailyUploadJob-context.xml
    2010-06-28 15:44:31,923 DEBUG BatchJobRestartHelper  - checkRestartJob()-->START
    2010-06-28 15:44:32,048 DEBUG BatchJobRestartHelper  - Job Instance for Job itemMasterDailyUploadJob = JobInstance: id=535, JobParameters=[{jobOrderId=222, type=monthly}], Job=[itemMasterDailyUploadJob]
    2010-06-28 15:44:32,064 DEBUG BatchJobRestartHelper  - Job Execution for Job itemMasterDailyUploadJob = JobExecution: id=659, startTime=2010-06-28 15:21:58.343,endTime=2010-06-28 15:25:33.687, lastUpdated=2010-06-28 15:25:33.687, status=FAILED, exitStatus=exitCode=FAILED;exitDescription=, job=[JobInstance: id=535, Job
    Parameters=[{jobOrderId=222, type=monthly}], Job=[itemMasterDailyUploadJob]]
    2010-06-28 15:44:32,064 INFO  BatchJobRunner  - Job = itemMasterDailyUploadJob failed on last instance.
    2010-06-28 15:44:32,064 INFO  BatchJobRunner  - Restarting the Job = itemMasterDailyUploadJob, with the Execution Id = 659
    2010-06-28 15:44:32,189 INFO  BatchJobExecutionListener  - Starting Job = itemMasterDailyUploadJob
    2010-06-28 15:44:32,220 DEBUG AttributeUtils  - get attribute for dbLogFile with the value of null
    2010-06-28 15:44:32,220 ERROR AbstractJob  - Encountered fatal error executing job
    org.springframework.batch.core.JobExecutionException: Flow execution ended unexpectedly
            at org.springframework.batch.core.job.flow.FlowJob.doExecute(FlowJob.jav
    a:141)
    
            at org.springframework.batch.core.job.AbstractJob.execute(AbstractJob.ja
    va:281)
            at org.springframework.batch.core.launch.support.SimpleJobLauncher$1.run
    (SimpleJobLauncher.java:120)
            at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecut
    or.java:49)
            at org.springframework.batch.core.launch.support.SimpleJobLauncher.run(S
    impleJobLauncher.java:114)
            at org.springframework.batch.core.launch.support.SimpleJobOperator.resta
    rt(SimpleJobOperator.java:267)
            at com.sears.hs.commons.batch.core.BatchJobRunner.startJob(BatchJobRunne
    r.java:129)
            at com.sears.hs.commons.batch.core.BatchJobRunner.main(BatchJobRunner.ja
    va:59)
    Caused by: org.springframework.batch.core.job.flow.FlowExecutionException: Ended flow=itemMasterDailyUploadJob at state=itemMasterDailyUploadJob.decision1 with exception
            at org.springframework.batch.core.job.flow.support.SimpleFlow.resume(Sim
    pleFlow.java:152)
            at org.springframework.batch.core.job.flow.support.SimpleFlow.start(Simp
    leFlow.java:124)
            at org.springframework.batch.core.job.flow.FlowJob.doExecute(FlowJob.jav
    a:135)
            ... 7 more
    Caused by: java.lang.NullPointerException
            at java.io.FileInputStream.<init>(Unknown Source)
            at java.io.FileInputStream.<init>(Unknown Source)
            at com.sears.hs.aim.pss.item.job.LogfileReaderDecider.checkForErrors(Log
    fileReaderDecider.java:40)
            at com.sears.hs.aim.pss.item.job.LogfileReaderDecider.decide(LogfileRead
    erDecider.java:30)
            at org.springframework.batch.core.job.flow.support.state.DecisionState.h
    andle(DecisionState.java:43)
            at org.springframework.batch.core.configuration.xml.SimpleFlowFactoryBea
    n$DelegateState.handle(SimpleFlowFactoryBean.java:141)
            at org.springframework.batch.core.job.flow.support.SimpleFlow.resume(Sim
    pleFlow.java:144)
            ... 9 more

    Code:
    <batch:job id="itemMasterUploadJob" parent="simpleJob">
                    <batch:step id="moveLawsonFileStep" parent="taskletStep" next="movePartCatalogFileStep">
    			<batch:tasklet ref="fileUtilTasklet" />
    		</batch:step>		
    		<batch:step id="movePartCatalogFileStep" parent="taskletStep" next="moveServigisticsFileStep">
    			<batch:tasklet ref="fileUtilTasklet" />
    		</batch:step>		
    		<batch:step id="moveServigisticsFileStep" parent="taskletStep" next="sqlLoadLawsonFileStep">
    			<batch:tasklet ref="fileUtilTasklet" />
    		</batch:step>
    		<batch:step id="sqlLoadLawsonFileStep" parent="taskletStep" next="decision1">
    			<batch:tasklet ref="sQLLoaderTasklet" />
    		</batch:step>		
    		<batch:decision decider="decider" id="decision1">
    			<batch:next on="FAILED" to="alertStep" />
    			<batch:next on="COMPLETED" to="sqlLoadPartCatalogFileStep" />
    		</batch:decision> 
    		<batch:step id="sqlLoadPartCatalogFileStep" parent="taskletStep" next="decision2">
    			<batch:tasklet ref="sQLLoaderTasklet" />
    		</batch:step>
    		<batch:decision decider="decider" id="decision2">			
    			<batch:next on="FAILED" to="alertStep"/>
    			<batch:next on="COMPLETED" to="sqlLoadServigisticsFileStep" />
    		</batch:decision>
    		<batch:step id="sqlLoadServigisticsFileStep" parent="taskletStep" next="decision3">
    			<batch:tasklet ref="sQLLoaderTasklet" />
    		</batch:step>
    		<batch:decision decider="decider" id="decision3">
    			<batch:next on="FAILED" to="alertStep"/>
    			<batch:next on="COMPLETED" to="insertIntoItemMasterStep" />
    		</batch:decision>
    		<batch:step id="insertIntoItemMasterStep" parent="taskletStep">
    			<batch:tasklet ref="itemMasterDAO" />
    		</batch:step>
                    <batch:step id="alertStep" parent="taskletStep">
    			<batch:tasklet ref="alertTasklet" />
    		</batch:step> 
    </batch:job>

  • #2
    This could be simply an implementation detail, so it might be possible to support your required behaviour without persisting the decision data, but it certainly doesn't work that way right now. All data for the decision has to be persisted so it can be replayed in the case of a restart (precisely your situation). You probably need to put them or a memento for them in the JobExecution context.

    Comment


    • #3
      JobExecutionException

      Hello Dave,

      I'm using Microsoft SQL Server 2008.
      It does not support "Sequence" creation and only allows one TimeStamp data-type per table.
      FYI : I have refered the table creation creation scripts.
      Due to these constraints,I'm unable to create Spring-Batch framework tables for persisting Job details.
      Question : Can I use Db2 for Spring Framework tables and SQL Server for application purpose.
      Please provide configuration to do so.
      Exception encountered while executing program is :-
      10:21:14,765 ERROR SimpleAsyncTaskExecutor-1 AbstractJob:274 - Encountered fatal error executing job
      org.springframework.batch.core.JobExecutionExcepti on: Flow execution ended unexpectedly
      at org.springframework.batch.core.job.flow.FlowJob.do Execute(FlowJob.java:110)
      at org.springframework.batch.core.job.AbstractJob.exe cute(AbstractJob.java:250)
      at org.springframework.batch.core.launch.support.Simp leJobLauncher$1.run(SimpleJobLauncher.java:110)
      at org.springframework.core.task.SimpleAsyncTaskExecu tor$ConcurrencyThrottlingRunnable.run(SimpleAsyncT askExecutor.java:229)
      at java.lang.Thread.run(Thread.java:736)
      Caused by: org.springframework.batch.core.job.flow.FlowExecut ionException: Ended flow=dcjob at state=dcsplit with exception
      at org.springframework.batch.core.job.flow.support.Si mpleFlow.resume(SimpleFlow.java:148)
      at org.springframework.batch.core.job.flow.support.Si mpleFlow.start(SimpleFlow.java:124)
      at org.springframework.batch.core.job.flow.FlowJob.do Execute(FlowJob.java:105)
      ... 4 more
      Caused by: java.util.concurrent.ExecutionException: org.springframework.batch.core.job.flow.FlowExecut ionException: Ended flow=dcsplit#4 at state=step_Batch_ABC with exception
      at java.util.concurrent.FutureTask$Sync.innerGet(Futu reTask.java:233)
      at java.util.concurrent.FutureTask.get(FutureTask.jav a:94)
      at org.springframework.batch.core.job.flow.support.st ate.SplitState.handle(SplitState.java:101)
      at org.springframework.batch.core.job.flow.support.Si mpleFlow.resume(SimpleFlow.java:144)
      ... 6 more
      Caused by: org.springframework.batch.core.job.flow.FlowExecut ionException: Ended flow=dcsplit#4 at state=step_Batch_ABC with exception
      at org.springframework.batch.core.job.flow.support.Si mpleFlow.resume(SimpleFlow.java:148)
      at org.springframework.batch.core.job.flow.support.Si mpleFlow.start(SimpleFlow.java:124)
      at org.springframework.batch.core.job.flow.support.st ate.SplitState$1.call(SplitState.java:82)
      at org.springframework.batch.core.job.flow.support.st ate.SplitState$1.call(SplitState.java:80)
      at java.util.concurrent.FutureTask$Sync.innerRun(Futu reTask.java:314)
      at java.util.concurrent.FutureTask.run(FutureTask.jav a:149)
      ... 2 more
      Caused by: java.util.ConcurrentModificationException
      at java.util.LinkedHashMap$AbstractMapIterator.checkC oncurrentMod(LinkedHashMap.java:136)
      at java.util.LinkedHashMap$AbstractMapIterator.makeNe xt(LinkedHashMap.java:141)
      at java.util.LinkedHashMap$KeyIterator.next(LinkedHas hMap.java:201)
      at java.util.HashSet.writeObject(HashSet.java:193)
      at sun.reflect.GeneratedMethodAccessor21.invoke(Unkno wn Source)
      at <unknown class>.<unknown method>(Unknown Source)
      at java.io.ObjectStreamClass.invokeWriteObject(Object StreamClass.java:1015)
      at <unknown class>.<unknown method>(Unknown Source)
      at <unknown class>.<unknown method>(Unknown Source)
      at <unknown class>.<unknown method>(Unknown Source)
      at <unknown class>.<unknown method>(Unknown Source)
      at <unknown class>.<unknown method>(Unknown Source)
      at <unknown class>.<unknown method>(Unknown Source)
      at <unknown class>.<unknown method>(Unknown Source)
      at <unknown class>.<unknown method>(Unknown Source)
      at <unknown class>.<unknown method>(Unknown Source)
      at <unknown class>.<unknown method>(Unknown Source)
      at <unknown class>.<unknown method>(Unknown Source)
      at java.io.ObjectOutputStream.writeObject(ObjectOutpu tStream.java:347)
      at org.apache.commons.lang.SerializationUtils.seriali ze(SerializationUtils.java:107)
      at org.apache.commons.lang.SerializationUtils.seriali ze(SerializationUtils.java:132)
      at org.springframework.batch.core.repository.dao.MapS tepExecutionDao.copy(MapStepExecutionDao.java:51)
      at org.springframework.batch.core.repository.dao.MapS tepExecutionDao.updateStepExecution(MapStepExecuti onDao.java:92)
      at org.springframework.batch.core.repository.support. SimpleJobRepository.update(SimpleJobRepository.jav a:167)
      at org.springframework.batch.core.step.AbstractStep.e xecute(AbstractStep.java:185)
      at org.springframework.batch.core.job.AbstractJob.han dleStep(AbstractJob.java:348)
      at org.springframework.batch.core.job.flow.FlowJob.ac cess$100(FlowJob.java:43)
      at org.springframework.batch.core.job.flow.FlowJob$Jo bFlowExecutor.executeStep(FlowJob.java:137)
      at org.springframework.batch.core.job.flow.support.st ate.StepState.handle(StepState.java:60)
      at org.springframework.batch.core.job.flow.support.Si mpleFlow.resume(SimpleFlow.java:144)
      ... 7 more
      Please suggest.

      My usecase :-
      Is to execute multiple stored Procedures with varying input data.

      Configuration used :-

      <bean id="jobExplorer"
      class="org.springframework.batch.core.explore.supp ort.JobExplorerFactoryBean"
      p:dataSource-ref="dataSource" />

      <bean id="dataSource" destroy-method="close" class="org.apache.commons.dbcp.BasicDataSource">
      <property name="driverClassName" value="com.microsoft.sqlserver.jdbc.SQLServerDrive r"/>
      <property name="url" value="jdbc:sqlserver://:;databaseName="/>
      <property name="username" value=""/>
      <property name="password" value=""/>
      <property name="initialSize" value="50"/>
      </bean>

      <bean id="transactionManager" class="org.springframework.jdbc.datasource.DataSou rceTransactionManager">
      <property name="dataSource" ref="dataSource"/>
      </bean>

      <bean id="jobLauncher" class="org.springframework.batch.core.launch.suppo rt.SimpleJobLauncher">
      <property name="jobRepository" ref="jobRepository" />
      <property name="taskExecutor" ref="taskExecutor" />
      </bean>

      <bean id="taskExecutor" class="org.springframework.core.task.SimpleAsyncTa skExecutor" >
      <property name="concurrencyLimit" value="15"/>
      </bean>

      <bean id="jobRepository" class="org.springframework.batch.core.repository.s upport.SimpleJobRepository">
      <constructor-arg>
      <bean class="org.springframework.batch.core.repository.d ao.MapJobInstanceDao"/>
      </constructor-arg>
      <constructor-arg>
      <bean class="org.springframework.batch.core.repository.d ao.MapJobExecutionDao" />
      </constructor-arg>
      <constructor-arg>
      <bean class="org.springframework.batch.core.repository.d ao.MapStepExecutionDao"/>
      </constructor-arg>
      <constructor-arg>
      <bean class="org.springframework.batch.core.repository.d ao.MapExecutionContextDao"/>
      </constructor-arg>
      </bean>

      <bean id="partitioner" class="org.springframework.batch.core.partition.su pport.SimplePartitioner"/>

      <batch:job id="dcjob" restartable="true">
      <batch:split id="dcsplit" task-executor="taskExecutor">

      <batch:flow>
      <batch:step id="step_Batch_XYZ">
      <batch:tasklet ref="tasklet_Batch_XYZ" allow-start-if-complete="true" transaction-manager="transactionManager" />
      </batch:step>
      </batch:flow>

      <batch:flow>
      <batch:step id="step_Batch_ABC">
      <batch:tasklet ref="tasklet_Batch_ABC" allow-start-if-complete="true" transaction-manager="transactionManager" />
      </batch:step>
      </batch:flow>
      </batch:split>
      </batch:job>

      <bean id="tasklet_Batch_XYZ" parent="storedProcedureTasklet">
      <property name="storedProcedureName" value="XYZ"/>
      </bean>

      <bean id="tasklet_ABC" parent="storedProcedureTasklet">
      <property name="storedProcedureName" value="ABC"/>
      </bean>

      Comment

      Working...
      X