Announcement Announcement Module
Collapse
No announcement yet.
Multiple Resource Files for Job Execution Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • Multiple Resource Files for Job Execution

    I have requirement to process a flatfile through spring batch job execution. I get the list of files from the folder which needs to be processed. Please let me know how to accomplish this.

    For example: I have player.csv and player1.csv and player2.csv in a folder. I have these file names in a list. Please let me know if I can input all these files names for Jobparameters so that I can execute a single Job?

    Code:
    JobParameters jobParameters = new JobParametersBuilder().addString("input.file", "file:player.csv").toJobParameters();
    JobExecution execution = jobLauncher.run(job,jobParameters);
    assertEquals(ExitStatus.COMPLETED, execution.getExitStatus());

  • #2
    You can look at MultiResourcePartitioner

    configure a job that has a step. step has a partitioner. partitioner is MultiResourcePartitioner. You would set a job parameter that points to the directory containing the files to be processed. source.file is a regular expression.

    Code:
    	<job id="myJob" xmlns="http://www.springframework.org/schema/batch">
    	    <step id="start">
    	        <partition step="partitionedStep" partitioner="partitioner">
     	            <handler grid-size="${gridSize}" task-executor="taskExecutor"/>
    	        </partition>	    
    	    </step>
    
    	<step id="partitionedStep" xmlns="http://www.springframework.org/schema/batch">
    	    <tasklet>
    	        <chunk reader="itemReader" processor="itemProcessors" writer="itemWriter" />
    		</tasklet>			
    	</step>
    
    	<bean id="partitioner" class="org.springframework.batch.core.partition.support.MultiResourcePartitioner" scope="step">
    	    <property name="resources" value="#{jobParameters['source.dir']+jobParameters['source.file']}" /> 
    	</bean>
    
    	<bean id="itemReaderParent" class="org.springframework.batch.item.file.FlatFileItemReader" abstract="true">
    	    <property name="bufferedReaderFactory" ref="gzipReaderFactory" />
    		<property name="lineMapper" ref="logItemLineMapper" />
    	</bean>
    	
    
    	<bean id="itemReader" scope="step" autowire-candidate="false" parent="itemReaderParent">
    	    <property name="resource" value="#{stepExecutionContext[fileName]}" />
        </bean>
    
    	</job>

    Comment

    Working...
    X