Announcement Announcement Module
Collapse
No announcement yet.
Strange Skipping Listener Behaviour in Read and Write Operations Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • Strange Skipping Listener Behaviour in Read and Write Operations

    I have a job that reads a flat file of 11 records. I skip 1 item in reader and 4 in writers. I have one Skiplistener with SkipInRead() and SkipInWrite() method which simple logs the skip message in DB. Why my onSkipInRead is being called multiple times i.e. 3 times in this case which in result logs same message in DB 3 times. OnSkipinWrite works fine


    Code:
    <job id="ex_Job">
    		<step id="load">
    			<tasklet>
    				<chunk reader="reader" writer="writer" commit-interval="3" skip-limit="10">
    					<skippable-exception-classes>
    						<include class="org.springframework.batch.item.file.FlatFileParseException" />
    						<include class="org.springframework.batch.item.file.transform.IncorrectLineLengthException" />
    						<include class="org.springframework.dao.DataIntegrityViolationException"/>
    					</skippable-exception-classes>
    				</chunk>
    				<listeners>
    					<listener ref="genericSkipListener" />
    				</listeners>
    			</tasklet>
    		</step>
    	</job>
    Code:
    	
    public class GenericSkipListener extends AbstractBaseSkipListener implements
            SkipListener<FieldSet, FieldSet> {
    
        private static final Log LOGGER = LogFactory.getLog(GenericSkipListener.class);
    
        @Autowired
        private SkipedItemsDao skipedItemsDao;
    
    
        @Override
        public void onSkipInRead(final Throwable t) {
            LOGGER.warn("[" + getJobName() + "] onSkipInRead:" + t);
            skipedItemsDao.saveReadSkippedItem(createReadSkipedItem("READ", "", t));
        }
    
        @Override
        public void onSkipInWrite(final FieldSet item, final Throwable t) {
            LOGGER.warn("[" + getJobName() + "] onSkipWrite:" + t);
            skipedItemsDao.save(createSkipedItem("WRITE", item.toString(), t));
        }
    
        @Override
        public void onSkipInProcess(final FieldSet item, final Throwable t) {
            LOGGER.warn("[" + getJobName() + "] onSkipInProcess:" + t);
            skipedItemsDao.save(createSkipedItem("PROCESS", item.toString(), t));
        }
    	}
    My onSkipInRead() method is being call multiple times for the same item. Any idea guys ?

  • #2
    Hi, can you provide more information on "I skip 1 item in reader and 4 in writers."? How are you doing this may be that will help.

    Comment


    • #3
      Thanks Tiger.Spring for reply.

      In reader, the org.springframework.batch.item.file.FlatFileParseE xception is causing the skip of first record. It is a header record which has shorter lenght compared to standard file record.
      In Writer, org.springframework.dao.DataIntegrityViolationExce ption is causing the skip, as these records have the same key and Writer throws DuplicateKey exception.

      Comment


      • #4
        Any idea guys ?

        Comment


        • #5
          Strange skipping behaviour when commit-interval is not &quot;1&quot;

          I have a similar problem. My job reads a flatfile and writes into DB, if any exception occurs and a record is skipped its written to another flatfile. On running the job, in which there are records which causes "Duplicate entry 'XXX-YYYY' for key 'PPPPP' ", some records get inserted into both the DB and also into the response file (through the onSkipInWrite).

          Code:
          <chunk reader="myReader" processor="myValidatingProcessor" writer="myWriter" commit-interval="5" skip-limit="1000">
               <skippable-exception-classes>
          		<include class="org.springframework.batch.item.file.FlatFileParseException" />
          		<include class="org.springframework.batch.item.WriteFailedException" />
          		<include class="org.springframework.dao.DuplicateKeyException"/>
          		<include class="org.springframework.batch.item.validator.ValidationException"/>
               </skippable-exception-classes>
          </chunk>
          I have observed that this problem occurs only when the commit-interval="5". When I make it commit-interval="1", the code works properly.
          Is this the problem with Spring Batch? Or is there anything else which I should take care of?

          Comment

          Working...
          X