Announcement Announcement Module
Collapse
No announcement yet.
need help on defining a work flow using spring batch Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • need help on defining a work flow using spring batch

    I am very new to spring batch and trying to come up with a work flow using spring batch. I will be executing hadoop map/reduce job from each batch:step. Please find the attched work flow diagram. I have 2 approches to represent this work flow in the spring configuration file. Below is the most optimized way of executing but I am not able to identify how I can execute step number 10.

    Code:
    <batch:job id="basicJob">
    		<batch:step id="step1" next="split1">	
    		</batch:step>
    		<batch:split id="split1" task-executor="taskExecutor">
    			<batch:flow>
    				<batch:step id="step2" next="step4">
    				</batch:step>
    				<batch:step id="step4" next="split2">
    				</batch:step>
    				<batch:split id="split2" task-executor="taskExecutor">
    					<batch:flow>
    						<batch:step id="step6" next="step11">
    						</batch:step>
    						<batch:step id="step11" next="step12">
    						</batch:step>
    						<batch:step id="step12">
    						</batch:step>
    					</batch:flow>
    					<batch:flow>
    						<batch:step id="step7">
    						</batch:step>
    					</batch:flow>
    				</batch:split>
    			</batch:flow>
    			<batch:flow>
    				<batch:step id="step3" next="step5">
    				</batch:step>
    				<batch:step id="step5" next="split3">
    				</batch:step>
    				<batch:split id="split3" task-executor="taskExecutor">
    					<batch:flow>
    						<batch:step id="step8">
    						</batch:step>
    					</batch:flow>
    					<batch:flow>
    						<batch:step id="step9" next="split13">
    						</batch:step>
    						<batch:step id="step13" next="split14">
    						</batch:step>
    						<batch:step id="step14">
    						</batch:step>
    					</batch:flow>
    				</batch:split>
    			</batch:flow>
    		</batch:split>
    	</batch:job>
    Below representation executes everything well. But there is so munch unnecessary waiting before executing certain steps. For ex: step number 10 waits untill all of step11,step10 and step 12 finishes.

    Code:
    <batch:job id="basicJob">
    
    		<batch:step id="step1" next="split1">
    			
    		</batch:step>
    		<batch:split id="split1" task-executor="taskExecutor" next="split3">
    			<batch:flow>
    				<batch:step id="step2" >
    				</batch:step>
    			</batch:flow>
    			<batch:flow>
    				<batch:step id="step3" >
    				</batch:step>
    			</batch:flow>
    		</batch:split>
    		
    		<batch:split id="split3" task-executor="taskExecutor" next="split4">
    			<batch:flow>
    				<batch:step id="step4" >
    				</batch:step>
    			</batch:flow>
    			<batch:flow>
    				<batch:step id="step5" >
    				</batch:step>
    			</batch:flow>
    		</batch:split>
    		
    		<batch:split id="split4" task-executor="taskExecutor" next="split5">
    			<batch:flow>
    				<batch:step id="step6" >
    				</batch:step>
    			</batch:flow>
    			<batch:flow>
    				<batch:step id="step7" >
    				</batch:step>
    			</batch:flow>
    			<batch:flow>
    				<batch:step id="step8" >
    				</batch:step>
    			</batch:flow>
    			<batch:flow>
    				<batch:step id="step9" >
    				</batch:step>
    			</batch:flow>
    		</batch:split>
    		
    		<batch:split id="split5" task-executor="taskExecutor" next="split6">
    			<batch:flow>
    				<batch:step id="step10" >
    				</batch:step>
    			</batch:flow>
    			<batch:flow>
    				<batch:step id="step11" >
    				</batch:step>
    			</batch:flow>
    			<batch:flow>
    				<batch:step id="step12" >
    				</batch:step>
    			</batch:flow>
    		</batch:split>
    		<batch:split id="split6" task-executor="taskExecutor">
    			<batch:flow>
    				<batch:step id="step13" >
    				</batch:step>
    			</batch:flow>
    			<batch:flow>
    				<batch:step id="step14" >
    				</batch:step>Attachment 
    			</batch:flow>
    		</batch:split>
    
    	</batch:job>
    Can you please let me know whether there are any other solutions?.
    Attached Files

  • #2
    In your first snip, I don't see a step10 (or split10) so I'm not sure what part you are attempting to execute. In the second snip, I would expect step10, step11 and step12 to execute in parallel.

    Comment

    Working...
    X