Announcement Announcement Module
Collapse
No announcement yet.
spring batch exceptions are not skipped Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • spring batch exceptions are not skipped

    Hi, I'm new to spring batch. I'm trying to read pipeline seperated data from a flat file and write it to oracle db using FlatFileItemReader and IbatisBatchItemWriter. I'm facing an issue that i'm not able to skip any exceptions. Say, I have 12 records in the flat file and the 3rd record is corruped (as there are no proper pipeline seperation of data). I just want this record to be skipped and other records to be processed successfully. But only first two records are inserted to the db and the job is stopped. Could someone help on this issue please?

    My job config looks as follows:

    <batch:job id="job1" job-repository="simpleRepository" restartable="true">
    <batch:step id="step">
    <batch:tasklet transaction-manager="txManager"
    start-limit="100" >
    <batch:chunk reader="MultiResourceItemReader" processor="myitemprocessor" writer="ibatisDBWriter"
    commit-interval="1" skip-limit="250" skip-policy="skipPolicy" retry-limit="10">

    <batch:skippable-exception-classes>
    <batch:include class="java.lang.RuntimeException" />
    <batch:include class="java.sql.SQLSyntaxErrorException" />
    <batch:include class="java.lang.Exception" />
    <batch:include class="java.lang.NumberFormatException" />
    <batch:include class="java.lang.Throwable" />
    <batch:include class="org.springframework.batch.item.file.FlatFil eParseException"/>
    </batch:skippable-exception-classes>

    </batch:chunk>
    <batch:listeners>
    <batch:listener ref="mylistn"/>
    </batch:listeners>
    </batch:tasklet>



    </batch:step>
    </batch:job>

    <bean id="MultiResourceItemReader"
    class="org.springframework.batch.item.file.MultiRe sourceItemReader">
    <property name="resources" value="file:C:/myinputfiles/*textfile.txt " />
    <property name="delegate" ref="itemReader">
    </property>
    </bean>

    <bean id="itemReader" class="org.springframework.batch.item.file.FlatFil eItemReader">

    <!-- <property name="resource" value="classpath:textfile.txt" /> -->

    <property name="lineMapper">
    <bean class="org.springframework.batch.item.file.mapping .DefaultLineMapper">
    <property name="lineTokenizer">
    <bean
    class="org.springframework.batch.item.file.transfo rm.DelimitedLineTokenizer">
    <property name="delimiter" value="|" />
    <property name="names"
    value="xxxx_Number,xxxx_ID,transaction_ID,transact ion_date,transaction_time,ccccID,oooo_ID,amount" />
    </bean>
    </property>
    <property name="fieldSetMapper">
    <bean class="com.batch.extract.DataSetMapper" />
    </property>
    </bean>
    </property>
    </bean>



    <bean id="ibatisDBWriter"
    class="org.springframework.batch.item.database.Iba tisBatchItemWriter">
    <property name="assertUpdates" value="false" />
    <property name="sqlMapClient" ref="sqlMapClient" />
    <property name="statementId" value="ibatisDB.insert" />
    </bean>

    <bean id="myitemprocessor" class="com.batch.extract.MyItemProcessor">
    </bean>


    <bean id="mylistn" class="com.batch.extract.MyItemFailureListener">
    </bean>

    <bean id="simpleRepository" class="org.springframework.batch.core.repository.s upport.MapJobRepositoryFactoryBean">
    <property name="transactionManager" ref="txManager" />
    </bean>


    <bean id="skipPolicy" class="com.batch.extract.Skip_Policy">
    <property name="skipLimit" value="250"/>
    </bean>


    My skip policy java implementation as below: I guess the overridden shouldSkip(..) method is never invoked.


    public class Skip_Policy implements SkipPolicy {

    private int skipLimit;

    public void setSkipLimit(final int skipLimit) {
    this.skipLimit = skipLimit;
    System.out.println("\n\nskip limit is set by spring in custom skip policy..! ");
    }

    public boolean shouldSkip(final Throwable t, final int skipCount) throws SkipLimitExceededException {

    System.out.println("\n\ninside my overridden shouldSkip(....) method...!!");
    if (skipCount < this.skipLimit) {
    return true;
    }
    return false;
    }


    }


    My fieldsetmapper implementation looks as follows:
    DataSet is kind of my domain object.

    public class DataSetMapper implements FieldSetMapper<DataSet> {

    public DataSet mapFieldSet(FieldSet fs) {

    if (fs == null) {
    return null;
    }

    DataSet dataset = new DataSet();
    dataset.setStore_Number(fs.readString("xxxx_Number "));
    dataset.setRegister_ID(fs.readString("xxxx_ID"));
    dataset.setTransaction_ID(fs.readString("transacti on_ID"));
    dataset.setTransaction_date(fs.readString("transac tion_date"));
    dataset.setTransaction_time(fs.readString("transac tion_time"));
    dataset.setCustomerID(fs.readString("ccccID"));
    dataset.setOffer_ID(fs.readString("ooo_ID"));
    dataset.setAmount_saved(fs.readString("amount"));
    return dataset;
    }

    }

  • #2
    I am also facing the same problem. I configured the spring batch job with skip policy and it is throwing exception and stopped executing rest of the records. Any help will be appreciated.

    Comment


    • #3
      Originally posted by stalele View Post
      I am also facing the same problem. I configured the spring batch job with skip policy and it is throwing exception and stopped executing rest of the records. Any help will be appreciated.


      I could solve that issue by downloading and replacing the 2.0 jars with the latest spring batch 2.1 jars.

      Comment


      • #4
        Originally posted by jinnahonline View Post
        I could solve that issue by downloading and replacing the 2.0 jars with the latest spring batch 2.1 jars.
        I am using the 2.1 version, but I solved the problem by increasing the commit- interval to 50, I set it as 1 before. I do not know how that make sense to be honest but it works.

        Comment


        • #5
          I'm using spring-batch 2.1.8 and I'm not having a problem. I forced a ParseExeception and my job picks back up on the next line and runs till its finished.

          <batch:chunk reader="fixedLengthCustomerFileReader" writer="outputWriter" commit-interval="10" skip-limit="10">
          <batch:skippable-exception-classes>
          <batch:include class="org.springframework.batch.item.ParseExcepti on"/>
          </batch:skippable-exception-classes>

          </batch:chunk>

          Jeff

          Comment

          Working...
          X