Announcement Announcement Module
Collapse
No announcement yet.
Spring batch : MultiResourceItemReader Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • Spring batch : MultiResourceItemReader

    I want to use a org.springframework.batch.item.file.MultiResourceI temReader to read multiple XML files in a directory.

    When I declare my bean of type MultiResourceItemReader, I have to reference a "ressources" property. So I reference the folder where my XML are stored with a wildcard to include all files, this is OK. But I also have to reference a ItemReader, which I do be referencing a StaxEventItemReader. Here is my problem: When I declare my StaxEventItemReader bean, I have to reference a "ressource" property, if not Spring won't instantiate the bean. Is there a way to reference a ItemReader prototype without "resource" attribute that will be assigned by the MultiResourceItemReader when iterating over his "ressources" attribute?

    Here's a snippet:

    <bean id="multiResourceReader" class="org.springframework.batch.item.file.MultiRe sourceItemReader">
    <property name="resources" value="file:///mydata/data/*.xml" />
    <property name="delegate" ref="xmlFileReader" />
    </bean>

    <bean id="xmlFileReader" class="org.springframework.batch.item.xml.StaxEven tItemReader">
    <property name="fragmentRootElementName" value="orderNotification"/>
    <!--<property name="resource" ref="iDontKnow"/>-->
    <property name="fragmentDeserializer">
    <bean class="org.springframework.batch.item.xml.oxm.Unma rshallingEventReaderDeserializer">
    <constructor-arg>
    <bean class="org.springframework.oxm.xmlbeans.XmlBeansMa rshaller"/>
    </constructor-arg>
    </bean>
    </property>
    </bean>

  • #2
    I think what you have here is (http://jira.springframework.org/browse/BATCH-734). It was fixed in trunk, but not in 1.1.x yet. If it's a showstopper make a comment there and we can get a fix into the branch (maybe too late for 1.1.2 now).

    Comment


    • #3
      Dave,

      Per your request, I commented in Jira 734 on the availability of the fix. I sure do hope you can squeeze it into V1.1.2. My workaround is kludgey. Your fix is much more comprehensive.

      Thanks.

      GregA

      Comment


      • #4
        Thanks for the reply.

        My problem is not exactly the one described in Jira 734.

        Maybe I'm not using MultiResourceItemReader correctly? When I declare my MultiResourceItemReader bean and reference a folder with XML files in the "ressources" attribute it is OK. The problem is when I reference the "delegate" attribute. This attribute has to be an ItemReader and this ItemReader bean has to be declared with a "ressource" attribute.

        Here is what I don't understand: If I declare the "ressources" attribute in my MultiResourceItemReader bean, why do I have to assign a specific "ressource" in my ItemReader? I thought this would be done dynamically by the MultiResourceItemReader?

        Comment


        • #5
          BATCH-734 is the problem, trust me, and you are using MultiResourceItemReader correctly. The MultiResourceItemReader does indeed set the resource dynamically, but it never gets a chance up to 1.1.1 because of the null check on inistialisation of StaxEventItemReader.

          Comment


          • #6
            Thanks, looking forward to V1.1.2!

            Comment


            • #7
              Dave, Lucas:

              V1.1.2 of the MultiResourceItemReader works as expected. It also sets the delegate's resource as expected (I'm using FlatFileItemReader as the delegate and I no longer have to "dummy up" a resource for the FlatFileItemReader to get past the afterPropertiesSet() check).

              Thanks for squeezing this into V1.1.2. These changes help clean up my configuration file a bit.

              GregA
              Last edited by GregA; Aug 14th, 2008, 01:08 PM.

              Comment


              • #8
                Hi, everybody

                I am trying to use the multiresourcereader also.....
                For my case , if i didnt inject a dummy resource in the flatfileitemreader, i will hit the same error

                PHP Code:
                Caused byjava.lang.IllegalArgumentExceptionInput Resource must not be null
                    at org
                .springframework.util.Assert.notNull(Assert.java:112)
                    
                at org.springframework.batch.item.file.FlatFileItemReader.doOpen(FlatFileItemReader.java:227
                this is my configuration
                PHP Code:
                <bean id="cv001BatchInsertStep"
                        
                class="org.springframework.batch.core.step.item.SimpleStepFactoryBean">
                        <
                property name="transactionManager" ref="transactionManager" />
                        <
                property name="streams" ref="cv001BatchReader" />
                        <
                property name="jobRepository" ref="jobRepository" />
                        <
                property name="itemReader" ref="multiResourceReader" />
                        <
                property name="itemWriter" ref="itemWriter" />
                    </
                bean>

                <
                bean id="cv001BatchReader" class="org.springframework.batch.item.file.FlatFileItemReader">
                        <
                property name="firstLineIsHeader" value="true" />
                        <
                property name="lineTokenizer" ref="cv001CompositeFixedLengthTokenizer" />
                        <
                property name="fieldSetMapper">
                            <
                bean class="CV001Mapper" >
                                <
                property name="dateFormat" value="yyyyMMdd hhmmssSSS" />
                            </
                bean>
                        </
                property>
                    </
                bean>

                <
                bean id="multiResourceReader" class="org.springframework.batch.item.file.MultiResourceItemReader">
                    <
                property name="resources" value="classpath:IP*.txt"/>
                    <
                property name="delegate" ref="cv001BatchReader" />
                  </
                bean
                I am using V 1.1.3but when trying to downgrade to 1.1.2, still got the same issue. Probably i have something wrong in my config ?




                any reply is greatly appreciated

                regards
                ballistic_realm
                Last edited by ballistic_realm; Dec 17th, 2008, 10:07 PM.

                Comment


                • #9
                  You shouldn't register the delegate reader as stream with the step. In your case the step calls open on the delegate before the MultiResourceItemReader has set the input resource and hence you get the exception. MultiResourceItemReader takes care of updating the executing context on behalf of the delegate, so restartability will still work.

                  Comment


                  • #10
                    Hi robert


                    you are right. i omitted the stream already

                    thanks

                    ballistic_realm

                    Comment


                    • #11
                      Just a little question, I tried to modify the sample XmlFunctionalTests in the spring batch package to make it work with my own xml files

                      So I modified :

                      Code:
                      	<bean id="itemReader" class="org.springframework.batch.item.xml.StaxEventItemReader">
                      		<property name="fragmentRootElementName" value="customer" />
                      		<property name="resource" value="data/iosample/input/input.xml" />
                      		<property name="unmarshaller" ref="customerCreditMarshaller" />
                      	</bean>
                      by

                      Code:
                        <bean id="itemReader" class="org.springframework.batch.item.file.MultiResourceItemReader">
                          <property name="resources" value="data/iosample/input/input*.xml" />
                          <property name="delegate" ref="xmlItemReader" />
                        </bean>
                      
                      
                      	<bean id="xmlItemReader" class="org.springframework.batch.item.xml.StaxEventItemReader">
                      		<property name="fragmentRootElementName" value="Cash" />
                      		<property name="unmarshaller" ref="apsysCashMarshaller" />
                      	</bean>
                      but the file describing the job is
                      Code:
                      	<job id="ioSampleJob">
                      		<step id="step1">
                      			<tasklet>
                      				<chunk reader="itemReader" writer="itemWriter" commit-interval="6"/>
                      			</tasklet>
                      		</step>
                      	</job>
                      and of course I get the error :

                      Code:
                      NoSuchBeanDefinitionException: No unique bean of type [org.springframework.batch.item.ItemReader] is defined: expected single matching bean but found 2: [itemReader, xmlItemReader]
                      I have two beans with the same type so the autowire is not possible.
                      I'm not familiar with xbeans, I'm used to the old notation bean = ...

                      It is possible to use the previous notation but without autowiring ?

                      Comment

                      Working...
                      X