Announcement Announcement Module
No announcement yet.
Start job via shell when SpringBatch app is running in separate JVM? Page Title Module
Move Remove Collapse
Conversation Detail Module
  • Filter
  • Time
  • Show
Clear All
new posts

  • Start job via shell when SpringBatch app is running in separate JVM?

    Hi there, I'm tossing around some ideas on how to use SpringBatch for the ETL portion of my project without having to run it in a web container. I'm not new to SpringBatch, I have already crated two successful projects with it however these were both running in a web container (Tomcat 6 and WebSphere 7). Now I want it all to run from a shell script in Linux.

    1. I have to ETL data from a database view to a separate database table.
    2. The time between runs is 15 minutes but this can change to be less than one minute apart.

    Option 1:
    From shell script start the job and pass in parameters for the current job run. This will start the JVM, instantiate my beans, start the job, and shutdown. The next run will have to do it all over again.
    Pros: Lots of examples to do this and each run is self contained in a JVM.
    Cons: It is very costly to start a JVM from my external scheduler so if this is kicked off every minute I can run into performance issues.

    Option 2:
    Start my SpringBatch app via command line in a separate JVM. Then from my external scheduler I start my shell script which will send a "request" to invoke/start a job.
    Pros: All my beans are already instantiated and my scheduler will only need to send a request to start a job. I believe this will give me much better performance with frequent runs.
    Cons: I can't find examples on how to start/invoke a job when it is already running in a separate JVM.

    Has anyone done something like this before? For my request I think I can setup a web service but I don't want to go through the overhead of setting this all up when the SpringBatch app and the shell script are going to be on the same machine. Is there anyone way to send a "request" to an already running SpringBatch app?

    Thanks in advance.

  • #2
    Okay, so I think I have an approach that might work, I'm going to start the POC now.

    I will start my SpringBatch application via command line and I will have my job launcher polling a database table for a new record. When the launcher sees the new record I will then trigger my job to start.

    On the other end, my scheduler will be a simple database insert with parameters needed for the particular job run.

    In short, the database will bridge the communication gap between my scheduler and my job.

    If anyone has any other suggestions I would be very much interested in hearing them.

    I threw away the following ideas:
    1. web service
    2. files
    3. http requests
    4. socket