Announcement Announcement Module
Collapse
No announcement yet.
Scheduling Spring Batch dynamically from database? Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • Scheduling Spring Batch dynamically from database?

    Hi, I have a Spring Batch application embedded in a web application, I need to have a scheduler that takes the programming of jobs executions from database, that means If i update the programming in database somehow reload the scheduler so that it reprograms the jobs executions.

    Is it possible to do this using spring batch and a scheduler (like quartz)?
    Thanks

  • #2
    Just to clarify, what I need is something like this:

    My batch project configured with basically 1 main Job and "n" tasks. The main Job is going to decide to do one of this things for every task (depending on task configuration):
    1- read from file and write in DB
    2- read from file and write in file (file transfer)
    2- read from DB and write in another DB
    3- read from DB and write in file

    The tasks are going to be the ones that have schedules, so the main job is going to execute the task's process depending on the schedule.

    Also I have an external system that is going to manage tasks configurations (Create new ones, update schedules) and store them in the custom database.

    So the complicated part is to update the schedule in my batch project in run time, getting the schedule from the custom database.

    Comment


    • #3
      not sure if your updating directly in the database or you have some sort of user interface that sits on top of the DB updates, but you can use the quartz API to update triggers based on the input:

      http://quartz-scheduler.org/document.../UpdateTrigger

      Comment


      • #4
        Originally posted by Chris Schaefer View Post
        not sure if your updating directly in the database or you have some sort of user interface that sits on top of the DB updates, but you can use the quartz API to update triggers based on the input:

        http://quartz-scheduler.org/document.../UpdateTrigger
        yes I have an external web application with user interface that is is going to be the one that update the schedule of a job. But what I'm trying to do is to is to read that configuration from my web application (the one that contains spring batch and quartz) when my app is deployed and whenever the schedule changes.

        I'll give it a try with UpdateTrigger
        Thanks Chris,

        Comment


        • #5
          Im currently having some trouble getting quartz to trigger batch jobs inside my web container.

          I added another servlet besides the one that manages the mvc to put the scheduler to work when i start up the application. In my application context I imported 2 xml, one with batch infrastructure and another for batch jobs.

          This is in my web.xml
          Code:
                  <listener>
          		<listener-class>org.springframework.web.context.ContextLoaderListener</listener-class>
          	</listener>
          
          	<servlet>
          		<servlet-name>springmvc</servlet-name>
          		<servlet-class>org.springframework.web.servlet.DispatcherServlet</servlet-class>
          		<init-param>
          			<param-name>contextConfigLocation</param-name>
          			<param-value>/WEB-INF/springmvc-servlet.xml</param-value>
          		</init-param>
          		<load-on-startup>1</load-on-startup>
          	</servlet>
          
          	<servlet-mapping>
          		<servlet-name>springmvc</servlet-name>
          		<url-pattern>/</url-pattern>
          	</servlet-mapping>
          
          	<welcome-file-list>
          		<welcome-file>jsp/home.jsp</welcome-file>
          	</welcome-file-list>
          
          	<servlet>
          		<servlet-name>InitializeServlet</servlet-name>
          		<servlet-class>com.company.batch.quartz.InitializeServlet</servlet-class>
          		<load-on-startup>1</load-on-startup>
          	</servlet>
          spring-mvcservlet.xml
          Code:
          <?xml version="1.0" encoding="UTF-8"?>
          <beans xmlns="http://www.springframework.org/schema/beans"
          ...
          	<bean class="com.company.batch.web.JobLauncherController">
          		<constructor-arg ref="jobLauncher" />
          		<constructor-arg ref="jobRegistry" />
          	</bean>
          ...
          </beans>
          applicationContext.xml
          Code:
          <?xml version="1.0" encoding="UTF-8"?>
          <beans xmlns="http://www.springframework.org/schema/beans"
          	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
          	xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd">
          
          	<import resource="batch-infrastructure.xml"/>
          	<import resource="batch-jobs.xml"/>
          </beans>
          InitializeServlet.java
          Code:
          public class InitializeServlet extends HttpServlet {
          	public void init() throws ServletException {
          		try {
          			System.out.println("Initializing ");
          			CronScheduler objPlugin = new CronScheduler();
          		} catch (Exception ex) {
          			ex.printStackTrace();
          		}
          	}
          }
          CronScheduler.java
          Code:
          public class CronScheduler {
          	public CronScheduler() throws Exception {
          		SchedulerFactory sf = new StdSchedulerFactory();
          		Scheduler sche = sf.getScheduler();
          		sche.start();
          		JobDetail jDetail = new JobDetail("Newsletter", "NJob",SpringBatchQuartzJobLauncher.class);
          		CronTrigger crTrigger = new CronTrigger("cronTrigger", "NJob","0/59 * * * * ?");
          		sche.scheduleJob(jDetail, crTrigger);
          	}
          }
          SpringBatchQuartzJobLauncher.java
          Code:
          public class SpringBatchQuartzJobLauncher extends QuartzJobBean {
          	@Autowired
          	private JobLauncher jobLauncher;
          	private JobLocator jobLocator;
          	private JobExplorer jobExplorer;
          	private Map<String, String> jobParameters;
          	public static final String JOB_NAME = "holaMundoJob";
          	private static final Logger log = LoggerFactory.getLogger(SpringBatchQuartzJobLauncher.class);
          
          	@Override
          	@SuppressWarnings("unchecked")
          	protected void executeInternal(JobExecutionContext context) throws JobExecutionException {
          		Map<String, Object> jobDataMap = context.getMergedJobDataMap();
          		String jobName = (String) jobDataMap.get(JOB_NAME);
          		try {
          			Job job = jobLocator.getJob(jobName);
          			JobParameters allParams = translateParams(job, jobParameters);
          			jobLauncher.run(job, allParams);
          		} catch (Exception e) {
          			log.error("Could not execute job.", e);
          		}
          	}
          
          	private JobParameters translateParams(Job job, Map<String, String> params)
          			throws Exception {
          		JobParametersBuilder builder = new JobParametersBuilder();
          		JobParameters incrementedParams = getNextJobParameters(job);
          		for (Map.Entry<String, JobParameter> param : incrementedParams
          				.getParameters().entrySet()) {
          			builder.addParameter(param.getKey(), param.getValue());
          		}
          		for (Map.Entry<String, String> param : params.entrySet()) {
          			builder.addString(param.getKey(), param.getValue());
          		}
          		return builder.toJobParameters();
          	}
          
          	private JobParameters getNextJobParameters(Job job)
          			throws JobParametersNotFoundException {
          		String jobIdentifier = job.getName();
          		JobParameters jobParameters;
          		List<JobInstance> lastInstances = jobExplorer.getJobInstances(jobIdentifier, 0, 1);
          		JobParametersIncrementer incrementer = job
          				.getJobParametersIncrementer();
          		if (incrementer == null) {
          			throw new JobParametersNotFoundException(
          					"No job parameters incrementer found for job="
          							+ jobIdentifier);
          		}
          		if (lastInstances.isEmpty()) {
          			jobParameters = incrementer.getNext(new JobParameters());
          			if (jobParameters == null) {
          				throw new JobParametersNotFoundException(
          						"No bootstrap parameters found from incrementer for job="
          								+ jobIdentifier);
          			}
          		} else {
          			jobParameters = incrementer.getNext(lastInstances.get(0)
          					.getJobParameters());
          		}
          		return jobParameters;
          	}
          
          	public void setJobLauncher(JobLauncher jobLauncher) {
          		this.jobLauncher = jobLauncher;
          	}
          
          	public void setJobLocator(JobLocator jobLocator) {
          		this.jobLocator = jobLocator;
          	}
          
          	public void setJobParameters(Map<String, String> jobParameters) {
          		this.jobParameters = jobParameters;
          	}
          
          	public void setJobExplorer(JobExplorer jobExplorer) {
          		this.jobExplorer = jobExplorer;
          	}
          }
          But Im getting an exception when I try to start my job.


          Code:
          09:54:43.672 [Timer-2] INFO  org.quartz.utils.UpdateChecker - New Quartz update(s) found: 1.8.5 [http://www.terracotta.org/kit/reflector?kitID=default&pageID=QuartzChangeLog]
          09:54:50.005 [DefaultQuartzScheduler_QuartzSchedulerThread] DEBUG org.quartz.simpl.SimpleJobFactory - Producing instance of Job 'NJob.Newsletter', class=com.company.batch.quartz.SpringBatchQuartzJobLauncher
          09:54:50.014 [DefaultQuartzScheduler_Worker-1] DEBUG org.quartz.core.JobRunShell - Calling execute on job NJob.Newsletter
          09:54:50.023 [DefaultQuartzScheduler_Worker-1] ERROR c.a.b.q.SpringBatchQuartzJobLauncher - Could not execute job.
          java.lang.NullPointerException: null
          	at com.company.batch.quartz.SpringBatchQuartzJobLauncher.executeInternal(SpringBatchQuartzJobLauncher.java:38) ~[SpringBatchQuartzJobLauncher.class:na]
          	at org.springframework.scheduling.quartz.QuartzJobBean.execute(QuartzJobBean.java:86) [spring-context-support-3.0.5.RELEASE.jar:3.0.5.RELEASE]
          	at org.quartz.core.JobRunShell.run(JobRunShell.java:216) [quartz-1.8.4.jar:na]
          	at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:549) [quartz-1.8.4.jar:na]
          Can someone point me in the right direction, because I don't know what to look for.
          Thanks

          Comment


          • #6
            Hi, my problems with getting Spring Batch to work with quartz persisted so I change my approach.

            My configurations are based in this project (Spring Batch In Action source code)
            https://github.com/acogoluegnes/Spri...webapp/WEB-INF

            I got it working using Spring scheduler.

            Is Spring Scheduler a good choice?
            is it possible to change programmatically the job's schedule using Spring Scheduler?
            Can i load from a custom database my job's schedules?

            Thanks
            Last edited by rmma; Aug 2nd, 2013, 08:52 AM.

            Comment

            Working...
            X