Announcement Announcement Module
Collapse
No announcement yet.
How to pass data from one step to next step? Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • How to pass data from one step to next step?

    This is another newbie question.

    I need to pass data created in a item writer in one step to a tasklet configured in the next step. How do I do that? Thanks.

  • #2
    Did you try defining a singleton bean in spring?

    Comment


    • #3
      Thanks. Singleton bean should work, however, this would limit multi-threaded parallel batch processing.

      I found the answer towards the end of the reference documentation, the right way is to use execution context. I'll try it out.

      Comment


      • #4
        There still is one problem. Looks like @BeforeStep annotation only works for item writers, not for a custom Tasklet. How do I obtain reference to the execution context from within a custom Tasklet? My thinking is if execution context can be injected to item writers, it should also be available to Tasklets because Tasklets is a simpler form that combines reader, processor and writer.

        Advice? Thanks.

        Comment


        • #5
          Figured out and got it working. From callback public RepeatStatus execute(StepContribution contribution,
          ChunkContext chunkContext) throws Exception {

          via ChunkContext, you can get StepExecution and its context, JobExecution and its context.

          Comment


          • #6
            For others, the link to the section in the reference docs is: http://static.springsource.org/sprin...aToFutureSteps

            Also, @BeforeStep will work fine on any class. You just have to make sure that you register the bean as a listener in the xml (using the <listener/> element).

            Finally, just using a singleton bean isn't good enough in general because you will have problems if you need to restart. To make the job restartable, you need to use the ExecutionContext to save the data that is being passed so that you don't have to start the job from the very beginning if it fails on the receiving step.

            Comment


            • #7
              Hi, I have a requirement to do the following:
              1) read reference data from a component
              2) read a file which has the first record as the header and the rest as detail recs.
              3) do some processing
              4) write out to another components interface

              I use the ref data to transform header and detail data.

              I wrote my own tasklet to be able to read data from a component. I injected the key name shown below (refDataKey)

              public RepeatStatus execute(StepContribution arg0, ChunkContext arg1)
              throws Exception {

              StepContext stepContext = arg1.getStepContext();
              StepExecution stepExecution = stepContext.getStepExecution();
              ExecutionContext executionContext = stepExecution.getExecutionContext();

              executionContext.put(refDataKey, refDataList);

              return RepeatStatus.FINISHED;

              }

              the xml looks like this

              <step id="refdata" next="processFile">
              <tasklet ref="depotReferenceInformationTasklet">
              <listeners>
              <listener ref="promotionListener"/>
              </listeners>
              </tasklet>
              </step>
              <bean id="promotionListener" class="com.tnt.express.dynamicrpp.listener.MyExecu tionContextPromotionListener">
              <property name="keys" value="refDataKey"/>
              </bean>

              I want to use the refDataKey data within LineCallbackHandler when, I'm doing the header processing and within the Item Processor. But, can't find a way of reading the execution context

              I tried extending the ItemStreamSupport and overriding the open, without success

              Any ideas?

              Comment


              • #8
                First, using the promotion listener is unnecessary on a Tasklet Step since you can just as easily write to the Job's ExecutionContext directly. (This isn't recommended in a chunk-oriented step for other reasons).

                Second, using the open() method won't work because the data you are looking for is in the Job's ExecutionContext (it was put there by the promotion listener), not the Step's. Each Step has its own unique context, so the only way to connect them it through the Job's context. As the documentation shows, this can be accomplished with a BeforeStep method:
                Code:
                    @BeforeStep
                    public void getMyData(StepExecution stepExecution) {
                        JobExecution jobExecution = stepExecution.getJobExecution();
                        ExecutionContext jobContext = jobExecution.getExecutionContext();
                        this.someObject = jobContext.get("someKey");
                    }

                Comment


                • #9
                  I hav tried using the promotion listener but it fails to pass data from one step to another. What could be the problem? I am using springbatch 2.0.3.
                  This is my config


                  <batch:job id="Job">
                  <batch:listeners>
                  <batch:listener ref="appJobExecutionListener" />
                  </batch:listeners>
                  <batch:step id="Step1" next="Step2">
                  <batch:tasklet>
                  <batch:listeners>
                  <batch:listener ref="itemFailureLoggerListener" />
                  <batch:listener ref="promotionListener"/>
                  </batch:listeners>
                  <batch:chunk reader="ItemReader" writer="ItemWriter"
                  commit-interval="1000" />
                  </batch:tasklet>
                  </batch:step>
                  <batch:step id="Step2" next="Step3">
                  <batch:tasklet ref="callStoredProcTasklet">
                  <batch:listeners>
                  <batch:listener ref="promotionListener"/>
                  </batch:listeners>
                  </batch:tasklet>
                  </batch:step>
                  <batch:step id="Step3">
                  <batch:tasklet ref="fileMoveTasklet"/>
                  </batch:step>
                  </batch:job>

                  <bean id="promotionListener" class="org.springframework.batch.core.listener.Exe cutionContextPromotionListener" scope="step">
                  <property name="keys" >
                  <list>
                  <value>current.resource.name</value>
                  <value>current.resource.dir.path</value>
                  </list>
                  </property>
                  </bean>
                  Last edited by forgetarun; Dec 5th, 2009, 01:54 AM.

                  Comment


                  • #10
                    Can you post your code so that we might see what you're doing wrong?

                    (Be sure to use CODE tags)

                    Comment


                    • #11
                      Thanks for replying.

                      I know its sound weird but I tried to make this work for 4 hours but suddenly now its started working after adding 'strict' property to the promotionlistener for sometime. then again stopped working.

                      Here is my code.

                      In the first step of my job I am loading the values in the context.

                      Code:
                      public class MyMultiResourceItemReader extends MultiResourceItemReader {
                      
                      .
                      .
                      .	
                      	@Override
                      	public void open(ExecutionContext executionContext) throws ItemStreamException {
                      
                      	  super.open(executionContext);
                      
                      	  try {
                      		  if (getCurrentResource()!=null) 
                      		  {
                      			  executionContext.put("current.resource.dir.path", getCurrentResource().getFile().getParentFile());
                      			  executionContext.put("current.resource.name", getCurrentResource().getFilename());
                      		  }
                      		  else
                      		  {
                      			  executionContext.put("current.resource.dir.path", null);
                      			  executionContext.put("current.resource.name", null);
                      		  }
                      	  } catch (Exception e) {
                      	    logger.error(e.getMessage(), e);
                      	  }
                      	}
                      	
                      	  @Override
                      	  public void update(ExecutionContext executionContext) throws ItemStreamException {
                      	 
                      	    super.update(executionContext);
                      	    
                      	 
                      	    try {
                      	 
                      	      if (getCurrentResource()!=null) {
                      			  executionContext.put("current.resource.dir.path", getCurrentResource().getFile().getParentFile());
                      	          executionContext.put("current.resource.name", getCurrentResource().getFilename());
                      	      } else {
                      	    	  executionContext.put("current.resource.dir.path", null);
                      	          executionContext.put("current.resource.name", null);
                      	      }
                      	 
                      	    } catch (Exception e) {
                      	    	logger.error(e.getMessage(), e);
                      	    }
                      	 }
                      }
                      In the second step , am trying to get it from jobcontext

                      Code:
                      public class ProcessorTasklet implements Tasklet {
                      	
                      	
                      	private String fileName = null;
                      	
                      
                      public RepeatStatus execute(StepContribution arg0, ChunkContext arg1)
                      			throws Exception {
                      		
                      		StepContext stepContext = arg1.getStepContext();
                      		StepExecution stepExecution = stepContext.getStepExecution();
                      	    JobExecution jobExecution = stepExecution.getJobExecution();
                      		ExecutionContext jobContext = jobExecution.getExecutionContext();
                              this.fileName = (String) jobContext.get("current.resource.name");
                      	.
                      	.
                      	.
                      		return null;
                      	}
                      
                          @BeforeStep
                          public void getData(StepExecution stepExecution) {
                              JobExecution jobExecution = stepExecution.getJobExecution();
                              ExecutionContext jobContext = jobExecution.getExecutionContext();
                              this.fileName = (String) jobContext.get("current.resource.name");
                          }
                      }
                      this is my step 3,

                      Code:
                      public class FileMoveTasklet implements Tasklet {
                      
                      	@Override
                      	public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) throws Exception {
                      	
                      		File dirParent;
                      		
                      		StepContext stepContext = chunkContext.getStepContext();
                      		StepExecution stepExecution = stepContext.getStepExecution();
                      	       JobExecution jobExecution = stepExecution.getJobExecution();
                      		ExecutionContext jobContext = jobExecution.getExecutionContext();
                      	        dirParent = (File) jobContext.get("current.resource.dir.path");
                      		String fileNem = (String) jobContext.get("current.resource.name");
                      
                      .
                      .
                      .
                      
                      		return RepeatStatus.FINISHED;
                      	}
                      
                      }
                      Code:
                      	<bean id="promotionListener" class="org.springframework.batch.core.listener.ExecutionContextPromotionListener" scope="step">
                          	<property name="keys" >
                          	 <list>
                                   <value>current.resource.name</value>
                      			 <value>current.resource.dir.path</value>
                              </list>
                          	</property>
                          	<property name="strict" value="true"/>
                      	</bean>
                      Last edited by forgetarun; Dec 5th, 2009, 05:55 AM.

                      Comment


                      • #12
                        1) First, this is never a good idea to have in your code. You should remove it so that errors aren't swallowed.
                        Code:
                        } catch (Exception e) {
                             logger.error(e.getMessage(), e);
                        }
                        2) I agree that it doesn't make sense that 'strict=true' would change the behavior since all this flag does is indicate whether or not to throw an exception.

                        3) In the step 2 code, I'm not sure why you're getting the filename in execute() AND the @BeforeStep. You should remove that code from execute() and just use the @BeforeStep to retrieve values from the context. Do the same for step 3: create a @BeforeStep and remove the code from execute().

                        4) The xml seems ok, but you haven't included anything about registration. Have you registered this bean as a listener on step 1? Further, have you registered steps 2 and 3 as listeners to make sure that the @BeforeStep is called?

                        5) Finally, are you sure that step 1 is completing with status COMPLETED?

                        Comment


                        • #13
                          thanks again.
                          I will correct the code as suggested. I hav verified the step 1 is completed properly.
                          am new to spring batch , can you tell me on how do I register a bean as listener?

                          Comment


                          • #14
                            You use the <listener/> tag, as shown in the section on listeners (http://static.springsource.org/sprin...gStepExecution) and the section on passing data to future steps (http://static.springsource.org/sprin...aToFutureSteps).

                            Comment


                            • #15
                              thank you. I could nt get the listener thing but its working now.

                              Comment

                              Working...
                              X