Announcement Announcement Module
Collapse
No announcement yet.
Upon job restart, step doesn't honor skippable-exception-classes Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • Upon job restart, step doesn't honor skippable-exception-classes

    PROBLEM DESCRIPTION: Upon restarting a failed step, the reader (FlatFileItemReader) tries to parse the file from the beginning as it tries to reach the point from where it has to pick up after the last failure. Along the way, the reader invokes the lineTokenizer and the fieldSetMapper for each of the already processed records. This appears to be far too much work for no gain when the point is to quickly get to the last record read from the file that was successfully committed. Additionally, during this recovery phase, the parsing exercise triggers parsing exceptions but the batch framework doesn't honor the "skippable-exception-classes" instruction.

    JOB DESCRIPTION:
    Code:
        <job id="cdmFeedProcessingJob" parent="simpleJob">
            <step id="mainProcessingStep" parent="simpleStep">
                <tasklet>
                    <chunk reader="fileReader" writer="genericWriter" commit-interval="100" skip-limit="1000">
                        <skippable-exception-classes>
                            org.springframework.batch.item.file.FlatFileParseException
                        </skippable-exception-classes>
                    </chunk>
                </tasklet>
            </step>
        </job>
    
        <beans:bean id="fileReader" class="org.springframework.batch.item.file.FlatFileItemReader">
            <beans:property name="lineMapper">
                <beans:bean class="org.springframework.batch.item.file.mapping.DefaultLineMapper">
                    <beans:property name="fieldSetMapper">
                        <beans:bean class="org.springframework.batch.item.file.mapping.BeanWrapperFieldSetMapper">
                            <beans:property name="prototypeBeanName" value="myBean"/>
                            <beans:property name="strict" value="false"/>
                        </beans:bean>
                    </beans:property>
                    <beans:property name="lineTokenizer">
                        <beans:bean class="org.springframework.batch.item.file.transform.DelimitedLineTokenizer">
                            <beans:property name="delimiter" value="|"/>
                            <beans:property name="names" value="accountId, productSynonym, productSynonymType, localCurrencyCode, balanceType, quantity1, quantity2, quantity3"/>
                        </beans:bean>
                    </beans:property>
                </beans:bean>
            </beans:property>
            <beans:property name="resource" value="classpath:${app.feedFile}"/>
            <beans:property name="strict" value="true"/>
        </beans:bean>
    MY SCENARIO:
    • Using Spring Batch 2.0.3.
    • Flat file containing approx. 2 million records
    • Process failed (due to insufficient heap) after successfully processing 1,097,434 records
    • Out of 1,097,434 records, 5,834 records were skipped (due to the skippable-exception-classes instruction) and the remaining were written successfully.
    • Upon restarting the process, the step fails at record #343 with the following exception (which is the first record in the feed file that can't be parsed):
      Code:
      org.springframework.batch.item.file.FlatFileParseException: Parsing error at line: 343, input=[123456789           |X12345              |ABC|USD|HIC|300.000000||82.700000]
              at org.springframework.batch.item.file.mapping.DefaultLineMapper.mapLine(DefaultLineMapper.java:49)
              at org.springframework.batch.item.file.FlatFileItemReader.doRead(FlatFileItemReader.java:179)
              at org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader.read(AbstractItemCountingItemStreamItemReader.java:87)
              at org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader.jumpToItem(AbstractItemCountingItemStreamItemReader.java:77)
              at org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader.open(AbstractItemCountingItemStreamItemReader.java:153)
              at org.springframework.batch.item.support.CompositeItemStream.open(CompositeItemStream.java:98)
              at org.springframework.batch.core.step.item.ChunkMonitor.open(ChunkMonitor.java:107)
              at org.springframework.batch.item.support.CompositeItemStream.open(CompositeItemStream.java:98)
              at org.springframework.batch.core.step.tasklet.TaskletStep.open(TaskletStep.java:379)
              at org.springframework.batch.core.step.AbstractStep.execute(AbstractStep.java:195)
              at org.springframework.batch.core.job.AbstractJob.handleStep(AbstractJob.java:348)
              at org.springframework.batch.core.job.flow.FlowJob.access$0(FlowJob.java:1)
              at org.springframework.batch.core.job.flow.FlowJob$JobFlowExecutor.executeStep(FlowJob.java:135)
              at org.springframework.batch.core.job.flow.support.state.StepState.handle(StepState.java:60)
              at org.springframework.batch.core.job.flow.support.SimpleFlow.resume(SimpleFlow.java:144)
              at org.springframework.batch.core.job.flow.support.SimpleFlow.start(SimpleFlow.java:124)
              at org.springframework.batch.core.job.flow.FlowJob.doExecute(FlowJob.java:103)
              at org.springframework.batch.core.job.AbstractJob.execute(AbstractJob.java:250)
              at org.springframework.batch.core.launch.support.SimpleJobLauncher$1.run(SimpleJobLauncher.java:110)
              at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:49)
              at org.springframework.batch.core.launch.support.SimpleJobLauncher.run(SimpleJobLauncher.java:105)
              at org.springframework.batch.core.launch.support.CommandLineJobRunner.start(CommandLineJobRunner.java:291)
              at org.springframework.batch.core.launch.support.CommandLineJobRunner.main(CommandLineJobRunner.java:448)
    Basically, not only is the reader trying to fully parse the feed file from the beginning during the recovery phase, the batch framework is unable to see that the exception being caused during the parsing should be skipped. The end result is that the job simply can't restart.

  • #2
    Maybe you could raise a JIRA?

    Comment


    • #3
      Just created an issue on the issue tracker:
      http://jira.springframework.org/browse/BATCH-1418

      Comment


      • #4
        I am first time here on this forum and it was really a nice experience

        Comment


        • #5
          Its really appreciative work and I hope you will keep it up as well

          Comment

          Working...
          X