Announcement Announcement Module
Collapse
No announcement yet.
Hosting Reusable Batch Jobs in a web container Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • Hosting Reusable Batch Jobs in a web container

    We were early adopters of Spring Batch, back in the 1.0 and 1.1 days. We have a long-running web container that hosts our dozen or so Batch jobs. In those days, there were two pieces of advice that perhaps we can now ignore as we are upgrading to the latest 2.1.6.

    - Don't reuse jobs, they are stateful, always create new instances for a new job. As a result, we used Dave Syer's ClassPathXmlApplicationContextJobFactory that we found in the examples, that reads in the job-specific application context, instantiates it, pulls the job from it, and launches it, and then throws away the job and the context when the job finishes. When the job runs again, the process is repeated. It seems that in the latest versions, this is no longer necessary - a job instance in the Spring context can be safely launched multiple times. This would simplify our launching configs greatly. Is this true?

    - Always use JTA, since the Batch execution context tables are updated in the same transaction as the read/write operations. Recently, I don't see references to JTA - is it now safe to use local transactions?

    I've looked through the documentation (and got a copy of the upcoming book) but I couldn't find any specific mention of these changes. I just wanted to confirm that these are true?

    Thanks!

  • #2
    - Don't reuse jobs, they are stateful, always create new instances for a new job. As a result, we used Dave Syer's ClassPathXmlApplicationContextJobFactory that we found in the examples, that reads in the job-specific application context, instantiates it, pulls the job from it, and launches it, and then throws away the job and the context when the job finishes. When the job runs again, the process is repeated. It seems that in the latest versions, this is no longer necessary - a job instance in the Spring context can be safely launched multiple times. This would simplify our launching configs greatly. Is this true?
    yes, you can safely run a job multiple times.

    - Always use JTA, since the Batch execution context tables are updated in the same transaction as the read/write operations. Recently, I don't see references to JTA - is it now safe to use local transactions?
    you would need JTA if your business tables and the batch tables aren't on the same database.

    Comment


    • #3
      alright, thank you!

      Comment


      • #4
        It could be me but I was also under the impression that the batch execution information was updated in its own transaction (and not as part of the enclosing business transaction) so that the business transaction can fail but you still see this failure in the batch tables.

        But ofcourse I could be wrongly informed or misunderstood something .

        Comment


        • #5
          @Marten, you are partly right. There has to be a separate transaction to record the final status of a Step, but on the other hand the quality of service guarantees (skip, rollback, retry, restart) all rely on the business transaction being shared with the JobRepository.

          Comment


          • #6
            What multiple jobs of the same type running at the same time? For example, if we are building search indexes for multiple sites using a single job that is parameterized (using JobParameters, site="site1" or site="site2"): can we run the two jobs concurrently? That is, Job buildSearchIndex(site="site1") and Job buildSearchIndex(site="site2") running at the same time.

            Won't they collide in the shared beans? The steps will have the same ids, helper beans will have the same ids, so that the two Jobs will use the same bean instances and they'll end up stomping on each others' data, no?

            In the old spin-up-a-new-child-ApplicationContext-for-each-job-execution, you didn't have these problems because a job's app context couldn't even see the other jobs' contexts, even if it was the same name.

            Comment


            • #7
              Originally posted by kdelong View Post
              Won't they collide in the shared beans?
              Not if they are step-scoped. That's what StepScope was invented for, but don't use it indiscriminately - it's only needed for stateful compoenents.

              Comment


              • #8
                Thanks very much Dave!

                Since we have over a dozen jobs, and many of the beans are named the same (copy-and-paste job creation), I actually went back to the launcher that creates a sub application context for each job. But I generified lot of our launching code to use the nicer apis in Batch 2 - everything is much cleaner now.

                Comment


                • #9
                  Originally posted by kdelong View Post
                  I actually went back to the launcher that creates a sub application context for each job.
                  There should be no need for that, but it doesn't hurt I guess. Name clashes are dealt with in Spring Batch 2.* with AutomaticJobRegistrar. I think it's covered in the user guide.

                  Comment

                  Working...
                  X