Announcement Announcement Module
Collapse
No announcement yet.
trouble with the MultiResourceItemReader... Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • trouble with the MultiResourceItemReader...

    I am trying to get the MultiResourceItemReader working and having some problems applying it...

    Here is the configuration for it (the input files sit in an "inputs" directory within the run directory):

    Code:
    <bean id="multiFileItemReader" class="org.springframework.batch.item.file.MultiResourceItemReader" scope="step">
                    <property name="resources" value="file:#{stepExecutionContext.fileName}/*.dat" />
                    <property name="delegate" ref="itemReader" />
            </bean>
    
            <bean id="itemReader" class="org.springframework.batch.item.file.FlatFileItemReader" scope="step">
                    <property name="lineMapper" ref="lineMapper" />
            </bean>
    but on execution I get:

    Code:
    2009-02-23 17:45:14,485 [taskExecutor-1] WARN  org.springframework.batch.item.file.MultiResourceItemReader  No resources to read
    2009-02-23 17:45:14,509 [taskExecutor-1] ERROR org.springframework.batch.core.step.AbstractStep  Exception while closing step execution resources
    org.springframework.batch.item.ItemStreamException: Error while closing item reader
            at org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader.close(AbstractItemCountingItemStreamItemReader.java:94)
            at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source)
            at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
            at java.lang.reflect.Method.invoke(Method.java:616)
            at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:310)
            at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:182)
            at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:149)
            at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:131)
            at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:119)
            at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
            at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
            at $Proxy9.close(Unknown Source)
            at org.springframework.batch.item.file.MultiResourceItemReader.close(MultiResourceItemReader.java:130)
            at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source)
            at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
            at java.lang.reflect.Method.invoke(Method.java:616)
            at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:310)
            at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:182)
            at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:149)
            at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:131)
            at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:119)
            at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
            at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
            at $Proxy6.close(Unknown Source)
            at org.springframework.batch.item.support.CompositeItemStream.close(CompositeItemStream.java:86)
            at org.springframework.batch.core.step.tasklet.TaskletStep.close(TaskletStep.java:331)
            at org.springframework.batch.core.step.AbstractStep.execute(AbstractStep.java:245)
            at org.springframework.batch.core.partition.support.TaskExecutorPartitionHandler$1.call(TaskExecutorPartitionHandler.java:108)
            at org.springframework.batch.core.partition.support.TaskExecutorPartitionHandler$1.call(TaskExecutorPartitionHandler.java:107)
            at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
            at java.util.concurrent.FutureTask.run(FutureTask.java:166)
            at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
            at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
            at java.lang.Thread.run(Thread.java:636)
    Caused by: java.lang.NullPointerException
            at org.springframework.batch.item.file.FlatFileItemReader.doClose(FlatFileItemReader.java:214)
            at org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader.close(AbstractItemCountingItemStreamItemReader.java:91)
            ... 33 more
    Also, if I might ask... which is the appropriate TaskExecutor to be using for the configuration proposed in section 7.4 of the docs http://static.springsource.org/sprin...alability.html?

    org.springframework.core.task.SimpleAsyncTaskExecu tor

    or

    org.springframework.scheduling.concurrent.ThreadPo olTaskExecutor

    Note: I am using local threads.

    Thanks.

    Keith

  • #2
    The delegate itemReader doesn't need to be step scoped. Does it make any difference if you change that? I tried the same thing and it worked for me.

    SimpleAsyncTaskExecutor is, well, "simple"; the ThreadPool* version has more properties. Look in the Spring user guide and Javadocs for more info.

    Comment


    • #3
      Resolved...

      The problem was just a bad case of the "Mondays"....

      Comment


      • #4
        I lied...

        I did have a case of the mondays but the problem is not resolved... I am not getting substitution for some reason:

        Code:
        2009-02-24 13:10:32,843 [taskExecutor-1] DEBUG org.springframework.batch.core.scope.util.PlaceholderTargetSource  Replaced [file:#{stepExecutionContext.fileName}/*.psv] with [file:#{stepExecutionContext.fileName}/*.psv]
        it seems to work if all files are in the run directory but not in the inputs directory... (i.e. my input variable is not getting substituted).

        Also...

        How can I validate that I am running correctly? if I put the files in the root directory it runs but I am having a little difficulty identifying the file:thread associations... is there anything I can look for to ensure I did things correctly?

        Thanks.

        Keith

        Comment


        • #5
          What do you mean by "file:thread associations"? You can try running in a single thread to make it simple?

          BTW I think you might find that #{stepExecutionContext.fileName} has to be #{stepExecutionContext[fileName]} until we get Spring 3.0 and the full EL.

          Comment


          • #6
            Well... that was one problem... (using the wrong syntax) but that syntax is written that way in the doc in section 7.4... I didn't realize it was not valid yet. The other problem was that I was assuming the stepExecutionContext would inherit the jobParameters by default but that doesn't seem to be the case... when I changed it to jobParameters... it worked.

            With regard to the file:thread associations... I am just trying to make sure I did everything correctly... I want to be sure I linked all the components properly... Its obvious that I have 16 partitions but I want to make sure each one is correctly processing a separate input file...

            if you look at this log snippet... are does it appear as though 16 threads are running simultaneously? Every log entry appears under the taskExecutor-1 thread which I am assuming is the master thread. Is that normal?

            Thanks.

            Keith

            Comment


            • #7
              I fixed the error in the docos, thanks.

              Your log isn't complete so I can't really tell what's happening. Is taskexecutor-1 the thread that you launched the Job in (through a JobLauncher)? How did you configure the TaskExecutor for the PartitionHandler?

              Comment

              Working...
              X