Announcement Announcement Module
Collapse
No announcement yet.
if Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • if

    <batch:job id="newsSimJob" >
    <batch:step id="news_sim_script_step" next="news_sim_hdp_step" >
    <batch:tasklet ref="news_sim_script_tasklet" />
    </batch:step>
    <batch:step id="news_sim_hdp_step" next="news_sim_cp2local_step" >
    <batch:tasklet ref="news_sim_tasklet" />
    </batch:step>
    <batch:step id="news_sim_cp2local_step" next="new_sim_redis_step" >
    <batch:tasklet ref="news_sim_cp2local_tasklet" />
    </batch:step>
    <batch:step id="new_sim_redis_step">
    <batch:tasklet ref="simOutputTask" />
    </batch:step>
    </batch:job>

    <hdp:job id="news_convert_sim_job"
    input-path="${sim.input.path}/news/length/" output-path="${sim.output.path}/news/convert/"
    mapper="com.xxx.wap.algorithm.mapred.sim.SimConver tJob.MapClass"
    reducer="com.xxx.wap.algorithm.mapred.sim.SimConve rtJob.Reduce"
    jar="${jar.file.path}"
    input-format="org.apache.hadoop.mapreduce.lib.input.Text InputFormat"
    output-format="org.apache.hadoop.mapreduce.lib.output.Seq uenceFileOutputFormat"
    map-key="org.apache.hadoop.io.Text"
    map-value="com.xxx.wap.algorithm.model.sim.SimKeyValue "
    key="org.apache.hadoop.io.Text"
    value="com.xxx.wap.algorithm.model.sim.SimKeyValue Set"
    configuration-ref="hadoopConfiguration"
    scope="prototype"
    />

    if i execute the job again,i must write this configuration again???how to create a different hadoop job with <hdp:job

  • #2
    Each step is represented by a definition. Spring Batch supports branches and loops - check out its documentation. If you want to reuse the same definition you can try to either define a template bean definition from which the steps inherit from and then modify only the relevant parts or use SpEL so that each time the step definition is evaluated, the appropriate results are returned.
    This should be as well covered by the Spring Batch manual.

    Comment

    Working...
    X