Announcement Announcement Module
Collapse
No announcement yet.
Extrange behaviour with CompositeWriter, commit-interval and SkipListener Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • Extrange behaviour with CompositeWriter, commit-interval and SkipListener

    Hello all,

    I'm having a problem in job. The basic structure of the job is:

    Code:
    <job id="cargaMasivaJob">
    		<step id="cargaMasivaStep">
    		 	<tasklet>
    				<chunk reader="myFileReader"  writer="compositeWriter" commit-interval="500" skip-limit="100000">
    					<streams>
    						<stream ref="okFileWriter"/>
    					<!-- stream skippedListener -->
                                                     <stream ref="errorWriter"/>
    					</streams>					 
     	 				<skippable-exception-classes>java.lang.Exception</skippable-exception-classes>
      				</chunk>
      				<listeners>
      					<!-- skipListener -->
      					<listener ref="itemFailureLoggerListener"/>
      				</listeners>
    			</tasklet>		
    		</step>
    	</job>
    I'm reading a file as input, the composite writer call a java service and a fileWriter (to write the ok entries). Also i have an skipLister that use another fileWriter to write the skipped entries (bad ones).

    Code:
    <beans:bean id="compositeWriter"  class="org.springframework.batch.item.support.CompositeItemWriter">
       		<beans:property name="delegates">
    	        <beans:list>
    	            <beans:ref bean="itemWriterAdapter" />
    	            <beans:ref bean="flatFileItemWriter" />
    	        </beans:list>
        	</beans:property>
    	</beans:bean>
    
    The flatFileItemWriter use in its definition this resource:
    <beans:bean id="outputResource"
    		class="org.springframework.core.io.FileSystemResource" scope="step">
    		<beans:constructor-arg value="ok.txt" />
    	</beans:bean>
    The error listener sounds like:
    Code:
    	<beans:bean id="itemFailureLoggerListener" class="es.indra.sigle.batch.util.listeners.StandardVOSkipListener">
    		<beans:property name="errorWriter" ref="errorWriter"/>
    	</beans:bean>
    	<!-- Writer del los errores y su outputResource correspondiente -->
    	<beans:bean id="errorWriter"
    		class="org.springframework.batch.item.file.FlatFileItemWriter">
             ......
    
    The errorWriter use a resource like (that's diferent files):
    <beans:bean id="outputErrorResource"
    		class="org.springframework.core.io.FileSystemResource" scope="step">
    		<beans:constructor-arg value="bad.txt" />
    	</beans:bean>
    The problem is when i specify a commit-interval="500" (to test transaction i'm using a number greater than the readed file), the framework writes in the ok.txt file all lines proccesed ok and then append the error ones, leaving the bad.txt file empty.

    If i specity a commit-interval="1" all it's working well. The ok file has all the information and de bad.txt the failed ones.

    Trying to debug the spring-batch classes, I was able to see that all is working well, it's trying to write into the correct file. When it's encounter the first bad record, it call the skip lister, and its write in the errorFileWriter.

    The problem come when the spring-batch it's calling transactionManager.commit(transaction); When it has finished, it has modified the files, leaving the bad.txt file empty, and the ok.txt with the mix of ok and bad lines.

    I don't know what is happening inside the transactionManager, but it isn't calling the FlatFileWriter write method. I don't know how it's accesing to the outputStream of the writer directly (and don't discrimine between the normal and the listener writer)

    Thank you very much and sorry for my bad english. If i don't explain well just tell my for a rewrite
    Last edited by acamara; Feb 25th, 2013, 11:17 AM.

  • #2
    There was a bug in 2.0 I think that meant you couldn't use two file writers in the same transaction. If you try 2.1 you might find that it is fixed.

    Comment

    Working...
    X