Announcement Announcement Module
Collapse
No announcement yet.
Slave machine is down in spring batch partition model Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • Slave machine is down in spring batch partition model

    Hi,

    Now I wanna use spring batch partition model to handle multiple big files concurrently. But now I have a case that job failed due to one slave machine is down, so the step on this slave machine stopped working. I know spring batch could be restarted to process the failed step, but when I attempt to restart, it failed due to the step status is still "Started". I guess this is due to spring batch stop running status in database, but machine failed, so status is no update on that time. Is that right?

    Any good suggestion on this?

    Really appreciate in advance.

    <bean:bean name="partitionStep" class="org.springframework.batch.core.partition.su pport.PartitionStep">
    <beanroperty name="partitionHandler">
    <bean:bean class="org.springframework.batch.core.partition.su pport.TaskExecutorPartitionHandler">
    <beanroperty name="step" ref="remoteStep" />
    <beanroperty name="gridSize" value="10" />
    <beanroperty name="taskExecutor" ref="taskExecutor" />
    </bean:bean>
    </beanroperty>
    <beanroperty name="stepExecutionSplitter">
    <bean:bean class="org.springframework.batch.core.partition.su pport.SimpleStepExecutionSplitter">
    <bean:constructor-arg ref="jobRepository" />
    <bean:constructor-arg value="true" />
    <bean:constructor-arg value="billLoadStep" />
    <bean:constructor-arg ref="multiResourcePartitioner" />
    </bean:bean>
    </beanroperty>
    <beanroperty name="jobRepository" ref="jobRepository" />
    </bean:bean>

    <bean:bean id="multiResourcePartitioner" class="org.springframework.batch.core.partition.su pport.MultiResourcePartitioner">
    <beanroperty name="keyName" value="fileName"/>
    <beanroperty name="resources" value="file:/opt/*.CSV"/>
    </bean:bean>

  • #2
    Consolidate to make it clear:

    Spring batch will store job running status in database, but when the machine is down, how to get the job or step status is updated correctly, such as "Failed", rather still "Started" in database?

    Comment


    • #3
      Two things here:
      1. If you are using an underlying communication channel that supports guaranteed delivery (JMS for example), if a partition is being worked on, then the box goes down, the message will be put back on the queue to be reprocessed so no additional work is required.
      2. To reset the job statuses per your second question, it would need to be done manually. There are no facilities within Spring Batch to clean up failed partition executions.

      Comment


      • #4
        Really appreciate

        Originally posted by mminella View Post
        Two things here:
        1. If you are using an underlying communication channel that supports guaranteed delivery (JMS for example), if a partition is being worked on, then the box goes down, the message will be put back on the queue to be reprocessed so no additional work is required.
        2. To reset the job statuses per your second question, it would need to be done manually. There are no facilities within Spring Batch to clean up failed partition executions.
        Thanks a lot for your info, and I will have a test locally.

        Comment

        Working...
        X