Announcement Announcement Module
Collapse
No announcement yet.
Options for reuse of job's/step's? Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • Options for reuse of job's/step's?

    I have several 'jobs' where i need to read data from n db tables and ouput to n files.
    The only variants are the query and db properties and the output filenames.

    This could be configured as a single job with n steps, but there's a lot of duplication of configuration.

    I could run n instances of the same job definition (maybe using groupName to differentiate the instances), but for me a job is quite expensive to set up - scheduler configuration is another team, and I'd have to build dependencies into my scheduler - all n 'job's' must succeed before transmission of the extracted n files (I do this myself as part of the last step at the moment), so I want to run all n extracts under a single Job.

    Also important is that is a recurring pattern - i often get new requirements that look like this pattern.

    My ideal solution is to be able to add new db-to-file steps to an existing job by just adding some .properties, and to be able to rollout a brand new job consisting of the
    n instances of the db-to-file step without any new spring-batch.xml - just a new property file.

    Can anyone suggest any options that might help to reduce config duplication/effort required?

  • #2
    Maybe you could run your n tasks as steps in a partition step? A Partitioner would create new input data for each step depending on its configuration via the properties file.
    Last edited by Dave Syer; Jul 22nd, 2011, 04:04 AM. Reason: spelling

    Comment

    Working...
    X