Announcement Announcement Module
Collapse
No announcement yet.
ItemReader commit not happenning Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • ItemReader commit not happenning

    I am experiencing a strange problem with Item Reader/Writer with Async Task Executor and SimpleStepFactoryBean.
    I tweaked the parallelJob (example) reader/writer implementation to suit my needs.
    I am able to run the job successfully only when I run in the debug mode. Also, this job is working fine when I deployed in a servlet container.

    I have set the 'throttleLimit' property a non-default value of 20 and when I removed that, the job worked fine.

    Is it a bug or because of incorrect usage?

    Thanks
    Last edited by patb23@gmail.com; Oct 27th, 2008, 08:52 AM. Reason: Edited for clarity

  • #2
    Can you provide a simple configuration and unit test that shows the issue? It could be some sort of crossed wire between the throttle limit and the task executor settings. Or your "tweaks" might have been introducing extra state that isn't thread safe?

    Comment


    • #3
      Yes the 'throttleLimit' is not working only with my code.

      My 'tweaks' are limited to changing only the SQL query and am attaching the file. In the writer, I am making a webservice call.

      These are the things I noticed while testing:
      1. I believe that task executor creates as many 'reader' thread as throttleLimit. All the threads that read the object from the DB are completing the writer method call.
      2. throttleLimit greater than 8 is failing.
      3. The threads are hanging even If I do nothing in the writer (I just placed logger statements in the write). Hence I guess writer is not introducing the problem.

      Please let me know, if you need any other details in helping me solve this.

      Thanks


      Code:
      	<bean id="simpleStep" class="org.springframework.batch.core.step.item.SimpleStepFactoryBean"
      		abstract="true">
      		<property name="transactionManager" ref="transactionManager" />
      		<property name="jobRepository" ref="mapJobRepository" />
      		<property name="startLimit" value="100" />
      		<property name="commitInterval" value="1" />
      	</bean>
      
      
      				<bean id="loading" parent="simpleStep">
      					<property name="throttleLimit" value="8"/>
      					<property name="taskExecutor">
                                               .....................

      Comment


      • #4
        I don't think you actually said what the problem is. How does it fail? If you can provide a unit test that would be perfect.

        Comment


        • #5
          As you said, It seems that cross-wired settings between SimpleStep and the TaskExecutor is the reason.

          I ran the 'ParallelJobFunctionTests' that came with the Spring Sample and I added only the throttleLimit property. I could see the same behavior.

          Anyways that I could specify the throttleLimit here?

          Code:
          				<bean id="loading" parent="simpleStep">
          					<property name="throttleLimit" value="9"/>
          					<property name="taskExecutor">
          						<bean
          							class="org.springframework.core.task.SimpleAsyncTaskExecutor">
          						</bean>	
          					</property>
          					<property name="itemReader">
          						<bean
          							class="org.springframework.batch.sample.item.reader.StagingItemReader">
          							<property name="lobHandler"
          								ref="lobHandler" />
          							<property name="dataSource"
          								ref="dataSource" />
          						</bean>
          					</property>
          					<property name="itemWriter">
          						<bean
          							class="org.springframework.batch.sample.item.writer.TradeWriter">
          							<property name="dao" ref="tradeDao" />
          						</bean>
          					</property>
          				</bean>

          Comment

          Working...
          X