Announcement Announcement Module
Collapse
No announcement yet.
How to limit number of jobs executing at the same time? Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • How to limit number of jobs executing at the same time?

    Hi,
    I'm building an batch application using Spring MVC, Batch and Quartz as scheduler. I need to limit the number of jobs that are running at the same time using a parameter that is in a database.

    For example:
    My parameter is 10.
    So if there are already 10 jobs running at a specific moment, when Quartz triggers the execution of another job, the Job doesn't run immediately, it waits until one of the 10 that are already running finishes, then runs.

    1. Is it possible to do this?
    2. Do I have to configure this behavior in Spring Batch or in Quartz?
    Thanks

  • #2
    Forget about getting the parameter from database, let's say it's in the Quartz Scheduler definition or the Spring's taskExecutor via xml configuration.

    This is mi Batch-infrastructure.xml
    Code:
    ...
            <bean id="transactionManager" class="org.springframework.batch.support.transaction.ResourcelessTransactionManager" />
    
    	<bean id="jobRepository" class="org.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean">
    		<property name="transactionManager" ref="transactionManager" />
    	</bean>
    
    	<bean id="taskExecutor" class="org.springframework.core.task.SimpleAsyncTaskExecutor" >
    	</bean>
    
    	<bean id="jobLauncher"  class="org.springframework.batch.core.launch.support.SimpleJobLauncher">
    		<property name="jobRepository" ref="jobRepository" />
    		<property name="taskExecutor" ref="taskExecutor" />
    	</bean>
    
    	<bean class="org.springframework.batch.core.configuration.support.JobRegistryBeanPostProcessor">
    		<property name="jobRegistry" ref="jobRegistry" />
    	</bean>
    
    	<bean id="jobRegistry" class="org.springframework.batch.core.configuration.support.MapJobRegistry" />
    ...
    an this is my Scheduling.xml

    Code:
    ...
    <bean id="scheduler" class="org.springframework.scheduling.quartz.SchedulerFactoryBean" ></bean>
    ...

    I've read something about the concurrencyLimit property of Spring's taskExecutor and Quartz's scheduler threadCount, wich of these can provide me with the functionality that I need.

    Basically have a X number of jobs allowed to run at the same time, and when another job tries to be executed, put it in a queue until some other job finishes.

    Thanks!

    Comment


    • #3
      Hi rmma, did you found a solution to this problem ? Thank you.

      Comment


      • #4
        Originally posted by cosmin.spinu View Post
        Hi rmma, did you found a solution to this problem ? Thank you.
        I haven't fully tested that part of my application, but I did it using a ThreadPoolTaskExecutor instead of theSimpleAsyncTaskExecutor. Then when my app runs (deployed) I get the parameter from database en set it to the taskExecutor using setMaxPoolSize and setCorePoolSize methods.

        Hope it helps.

        Comment


        • #5
          Why don't use a batch execution environment for that (bee is something like an application server for batches) ? Maybe https://code.google.com/p/jem-the-bee/ could help.

          Comment

          Working...
          X