Announcement Announcement Module
Collapse
No announcement yet.
Delay in Batch jobs Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • Delay in Batch jobs

    Hi,

    Am working in spring batch to call DB2 stored procedure jobs which flows like a network. say for example, A is the first job, B,C,D should execute parellely after the completion of A. F& G sholud execute parallely after the completion of D without waiting for B&C to complete. E should execute after the completion of Parallel jobs BCD. Like this the network flows. The problem i face is, execution time between each job is 8 to 10 secs. Which i want to reduce. Am running the application using Junit in local. The 10 secs delay is between A and BCD, not between B,C,D. Please expedite on the time delay and provide me the solution. Please find the below config details.

    <job id="myJob" job-repository="jobRepository" restartable="true">
    <step id="stepA" next="parallelProcessBCD">
    <tasklet ref="taskletjob" />
    </step>
    <split id="parallelProcessBCD" task-executor="taskExecutor"
    next="stepE">
    <flow>
    <step id="stepB">
    <tasklet ref="taskletjob" />
    </step>
    </flow>
    <flow>
    <step id="stepC">
    <tasklet ref="taskletjob" />
    </step>
    </flow>
    <flow>
    <step id="stepD" next="parallelFG">
    <tasklet ref="taskletjob" />
    </step>
    <split id="parallelFG" task-executor="taskExecutor" next="stepH">
    <flow>
    <step id="stepF">
    <tasklet ref="taskletjob" />
    </step>
    </flow>
    <flow>
    <step id="stepG">
    <tasklet ref="taskletjob" />
    </step>
    </flow>
    </split>
    <step id="stepH">
    <tasklet ref="taskletjob" />
    </step>
    </flow>
    </split>
    <step id="stepE" next="stepI">
    <tasklet ref="taskletjob" />
    </step>

    <step id="stepI">
    <tasklet ref="taskletjob" />
    </step>
    </job>
    </beans:beans>


    Hi all,

    Can anyone help me as soon as possible.

    Thanks,
    Tami
    Last edited by Tamizhselvi; Dec 8th, 2009, 02:45 AM.

  • #2
    There isn't realy much to go on there. How do you know that step A has finished when your 8-10 second pause starts? Is it possible that it is still running? What kind of task executor are you using for the split?

    Comment


    • #3
      Hi syed thanks for the reply.

      The task executor that i use here is "org.springframework.core.task.SimpleAsyncTaskExec utor"

      And i get to know about the job completion thro the schema file which upadtes the DB with start and end time of every step. And also i ensure the same through the StepExecutionListener Interface.Refer below.

      public class StepsExecutionListener implements StepExecutionListener {

      public ExitStatus afterStep(StepExecution arg0) {
      System.out.println("STEP ---- " + arg0.getId() + " End time "
      + new Date());

      return arg0.getExitStatus();
      }

      public void beforeStep(StepExecution arg0) {
      System.out.println("STEP ---- " + arg0.getId() + " Start time "
      + arg0.getStartTime());
      }

      }

      Comment


      • #4
        There's nothing in Spring Batch that would delay processing the next step. What platform do you deploy on? Maybe thread creation is controlled or slow (so SimpleAsyncTaskExecutor would notr be a good choice)?

        Comment


        • #5
          Hi Syer,

          Its a stand alone application and the backend is DB2. I used SimpleAsyncTaskExecutor since i need to run some steps parallely. In case that this is not the correct option, please suggest me wat shud i use to execute the steps in parallel.

          Thanks,
          Tami

          Comment


          • #6
            Syer,

            Please find the attachment for the step flow in a job.

            BCD are parallel. FG are parallel.
            FG should start after D completes, without waiting for B & C to complete.
            E should start once B,C,D completes.
            H should start after FG compltes without waiting for E to complete.
            I should start afetr both H,E completes.

            The propblem is the time constraint. This just a simple flow. Like this there many complex flows of jobs. Where we have scenarios like job dependencies.
            The jobs has to be executed in this particular flow and should not be time comsuming also. Please suggest me the how to implement this. You help will be appreciated.

            Thanks,
            Tami

            Comment


            • #7
              Hi Tamizhselvi,

              I do not know what is your experience with DB2. But in my opinion this is the conurrency problem in DB2. There is page locking (by default). It makes many problems with parallel processing if separated threads are dealing with the same tables.

              Keep the rules:
              - keep transactions short (frequently commit)
              - postpone changes as far as possible to end of transaction (e.g. use FlushMode.COMMIT if you are using Hibernate)

              Do you have -913 (timeouts/deadlocks) in the logs?
              Did you try to log times of preparation and execution sqls? (the best will be datasource level)

              HTH,
              Jul

              Comment

              Working...
              X