Announcement Announcement Module
Collapse
No announcement yet.
File Reading twice in UNIX Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • File Reading twice in UNIX

    Hi all,
    we are using spring batch 1.1.4 for our batch needs. In our spring batch application, we have around 4 files say

    xyz_f1
    xyz_F2
    xyz_f3
    xyz_f4

    we are using MultiResourceItemReader to read multiple files. spring batch reads all 4 files fine in Windows environment. However when we tried to execute the same spring batch program in UNIX environment, it skips the file xyz_f4 and reads the file xyz_f1 twice. This looks weird to us.

    Did anyone come across this? We are puzzled on how it reads the files twice in UNIX and not in Windows.


    Appreciate your help

    Thanks,
    Sundar

  • #2
    We do all our CI builds on Linux so I doubt if it is a platform issue. Can you provide a test case or some more details of what you did?
    Last edited by Dave Syer; Apr 30th, 2009, 04:46 AM. Reason: typo

    Comment


    • #3
      Hi,
      We are trying to run this batch in AIX UNIX environment. Here is the xml snippet we use for reading files.
      Code:
      <bean id="rateactionstep-BCC_CA" class="org.springframework.batch.core.step.item.SkipLimitStepFactoryBean">
      		<property name="transactionManager" ref="tm"/>
      		<property name="jobRepository" ref="jobRepository"/>
      		<property name="commitInterval" value="10"/>
      		<property name="skippableExceptionClasses" value="com.wellpoint.eas.rateaction.validator.SkippableValidationException"/>
      		<property name="fatalExceptionClasses" value="com.wellpoint.eas.rateaction.validator.NonSkippableValidationException"/>
      		<property name="skipLimit" value="10"/>				<property name="streams" ref="fileItemReader" />
      		<property name="itemReader" ref="validatingItemReader"/>
      		<property name="itemWriter" ref="transformingWriter_BCC_CA"/>
      		<property name="listeners" ref="stepListener" />
      	</bean>
      	<bean id="fileItemReader" parent="multiResourceReader"
      		autowire-candidate="false" />
      <bean id="flatFileItemReader" class="org.springframework.batch.item.file.FlatFileItemReader">
      <!-- 
      		<property name="resource" value="classpath:BCC_CA_032709_Monthly_F1.txt" />
       -->
      		<property name="lineTokenizer" ref="RateActionTokenizer"/>
      		<property name="fieldSetMapper" ref="rateActionFieldsetMapper"/>
      	</bean>
      
      	<bean id="validatingItemReader" class="org.springframework.batch.item.validator.ValidatingItemReader">
      		<property name="itemReader" ref="multiResourceReader"/> 
      		
      		<property name="validator" ref="itemValidator"/>
      	</bean> 
      
      	<bean id="itemValidator" class="com.wellpoint.eas.rateaction.validator.RateActionValidator"/>
      	<bean id="jobListener" class="com.wellpoint.eas.rateaction.listeners.JobListener">
      	<property name ="emailService" ref="emailService" />
      <property name ="rateActionAnniv" ref="rateActionAnniv"/>
      	<property name ="rateActionService" ref="rateActionService"/>
      	</bean>
      <bean id="multiResourceReader" class="org.springframework.batch.item.file.MultiResourceItemReader"> 
      <property name="resources" value="file:BCC_CA_*.txt"/>
      <property name="delegate" ref="flatFileItemReader"/>
      </bean>
      we have placed our four files in the our unix directory where we have our Jar files.

      BCC_CA_041009_M_070109_F2.txt
      BCC_CA_041009_M_070109_F5.txt
      BCC_CA_041009_M_070109_F6.txt
      BCC_CA_041009_M_070109_F1.txt

      The job reader F1, F2, F5 files and instead of reading F6 files, it again reads BCC_CA_041009_M_070109_F1.txt twice.

      I hope this helps you.

      Thanks,
      Sundar

      Comment


      • #4
        That looks pretty weird to me: the flatFileItemReader has a parent="multiResourceItemReader", which in turn has a delegate ref="flatFileItemReader". It's circular, so I'm not sure how to predict what should happen. I'm surprised it works at all. What happens if you ditch the parent="..." in the flatFileItemReader?

        Comment


        • #5
          Hi Dave,
          we figured out where the problem was

          In the xml,

          Code:
          <bean id="validatingItemReader" class="org.springframework.batch.item.validator.ValidatingItemReader">
          		<property name="itemReader" ref="multiResourceReader"/> 
          		
          		<property name="validator" ref="itemValidator"/>
          	</bean>
          It should have been
          Code:
          		<property name="itemReader" ref="flatfileItemReader"/>
          This worked fine both in Windows and UNIX environment. Yes, it was weird how it worked in windows.

          Thanks for your input

          Comment

          Working...
          X