Announcement Announcement Module
Collapse
No announcement yet.
Job Statistics Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • Job Statistics

    What are the various job statistics details provided by spring batch?
    Can i add additional statistical data apart from those spring batch provides?
    For eg: if i have a job has to reads 100 records from a table and it fails at 50th record. So when i run the job again, will it start from where it failed?
    Should this be handled programmatically or can spring batch handle this?

  • #2
    Spring batch keeps basic statistics about the run. For example, commitCount, rollbackCount, readSkipCount, itemCount, etc.

    The second part of your question is actually different from statistics. Storing state information for the purposes of starting where you left off again in the case of failure is what we call restart information. All state is stored in an ExecutionContext. When an ItemStream is opened, it is given the ExecutionContext for the step. If it's just starting, the context will be empty, if it's a restart, there will likely be state information in it from the previous run. (assuming something was put in it)

    The following section in the reference guide should help explain the concepts to you:

    http://static.springframework.org/sp...e.html#d0e2959

    Comment


    • #3
      Thank you for your reply. I have a few more doubts.
      1. Does spring batch provides any reporting feature? Suppose if i executed a job that has many steps and i want to see the job statistics report of the job in detail for each steps in the job?
      Or should i get the details from meta data schema tables programmatically?

      2. I could found out the commitCount and itemCount in metadata schema tables, but where other details like rollbackCount, readSkipCount etc will be stored?

      3. What are the different ways to stop a job? For eg: if i have a job with three steps. I invoked this job by a scheduler and i want to stop the job in the middle of execution of second step . Will the spring batch take care of finishing the second step execution properly? What will happen if i re-run the job? Will it start from first step again or from the third step?

      Comment


      • #4
        Originally posted by johny_z View Post
        1. Does spring batch provides any reporting feature?
        Not in 1.x, but as you said, you can always point any tool that understands database tables at the Spring Batch metadata. 2.0 has a JobExplorer interface (current name), which isn't really "reporting" (fancy formatting, aggregation, PDF etc.), but does gove you read-only access to the metadata.
        2. I could found out the commitCount and itemCount in metadata schema tables, but where other details like rollbackCount, readSkipCount etc will be stored?
        What version are you looking at? All those fields are stored in the step execution table in 1.1 and above.
        3. What are the different ways to stop a job?
        See the user guide section on launching a job (http://static.springframework.org/sp...n.html#d0e3476).
        For eg: if i have a job with three steps. I invoked this job by a scheduler and i want to stop the job in the middle of execution of second step . Will the spring batch take care of finishing the second step execution properly? What will happen if i re-run the job? Will it start from first step again or from the third step?
        It will stop where you ask it to (i.e. in the second step in your example). As long as your stateful components implement ItemStream (and are registered with the step) the second step will be completed on restart. See the user guide (same chapter, for example http://static.springframework.org/sp...n.html#d0e3476).

        Comment


        • #5
          Thanks a lot.
          Can a spring batch job be scheduled using Tivoli Worflow Scheduler?
          Is there any examples available for this?

          Comment


          • #6
            No reason why not, but you will have to do some work to get your stop and monitoring use cases working in 1.x. Most people using non-Java schedulers like Tivoli just use the CommandLineJobLauncher, which doesn't give you any access to the running JobExecution. 2.0 will make it easier (e.g. see JobOperator). In the meantime you would have to expose a remote endpoint (JMX, web service etc.) that could be contacted by Tivoli and keep n the a reference to the JobExecution internally in the server. The JMX sample does this.

            There are no samples using Tivoli per se (or any other proprietary tool), but the CommandLineJobLauncher is used extensively and has plenty of documentation. There is also a Websphere Performance Pack in the pipeline from SpringSource which probably will include Tivoli features. It's not available yet, and the feature set is not finalised, but contact [email protected] if you want to stay in the loop.
            Last edited by Dave Syer; Aug 25th, 2008, 06:56 AM. Reason: clarity

            Comment


            • #7
              Thank you.
              When i ran the QuartzBatchLauncher.java in the samples, i got the following error. But when executed adhocLoopJob.xml and footballJob.xml individually, both are executed properly.(I am using orcle db for batch meta-data tables instead of in-memory db; property files are there in the classpath also). Would you please help me out...

              Exception in thread "main" org.springframework.beans.factory.CannotLoadBeanCl assException: Cannot find class [${batch.lob.handler.class}] for bean with name 'lobHandler' defined in class path resource [data-source-context.xml]; nested exception is java.lang.ClassNotFoundException: ${batch.lob.handler.class}

              Originally posted by Dave Syer View Post
              No reason why not, but you will have to do some work to get your stop and monitoring use cases working in 1.x. Most people using non-Java schedulers like Tivoli just use the CommandLineJobLauncher, which doesn't give you any access to the running JobExecution. 2.0 will make it easier (e.g. see JobOperator). In the meantime you would have to expose a remote endpoint (JMX, web service etc.) that could be contacted by Tivoli and keep n the a reference to the JobExecution internally in the server. The JMX sample does this.

              There are no samples using Tivoli per se (or any other proprietary tool), but the CommandLineJobLauncher is used extensively and has plenty of documentation. There is also a Websphere Performance Pack in the pipeline from SpringSource which probably will include Tivoli features. It's not available yet, and the feature set is not finalised, but contact [email protected] if you want to stay in the loop.

              Comment


              • #8
                Thank you.
                When i ran the QuartzBatchLauncher.java in the samples, i got the following error. But when executed adhocLoopJob.xml and footballJob.xml individually, both are executed properly.(I am using orcle db for batch meta-data tables instead of in-memory db; property files are there in the classpath also). Would you please help me out...

                Exception in thread "main" org.springframework.beans.factory.CannotLoadBeanCl assException: Cannot find class [${batch.lob.handler.class}] for bean with name 'lobHandler' defined in class path resource [data-source-context.xml]; nested exception is java.lang.ClassNotFoundException: ${batch.lob.handler.class}

                Comment


                • #9
                  I got the following error when i ran QuartzBatchLauncher.java in the samples:
                  Exception in thread "main" org.springframework.beans.factory.CannotLoadBeanCl assException: Cannot find class [${batch.lob.handler.class}] for bean with name 'lobHandler' defined in class path resource [data-source-context.xml]; nested exception is java.lang.ClassNotFoundException: ${batch.lob.handler.class}

                  But when I ran "jobs/adhocLoopJob.xml" and "jobs/footballJob.xml" individually, both are executed successfully. I used oracle for meta-data tables instead of in-memory db. Property files are there in the classpath.

                  Would you please help me out...?

                  Comment


                  • #10
                    From batch-oracle.properties in the samples project:

                    Code:
                    batch.lob.handler.class=org.springframework.jdbc.support.lob.DefaultLobHandler
                    Did you include that file in your context (the easiest way is by setting a System property environment=oracle)?

                    Actually the lob handler is only there to support very old versions of Oracle, and we'll be removing it from the samples at some point anyway. If you have Oracle 10g or above you don't need it so you can root out the bean definition that is causing problems and delete it.

                    Comment


                    • #11
                      Thank you again. I am using Oracle 10g. I removed bean def for lobHandler and incrementerParent. Thus the previous problem solved.

                      But when i ran QuartzBatchLauncher, i am getting the following errors(some times) in between of job execution:
                      1.
                      org.springframework.batch.core.repository.NoSuchJo bException: No job configuration with the name [footballJob] was registered
                      2org.springframework.batch.core.repository.JobInst anceAlreadyCompleteException: A job instance already exists and is complete for parameters={}{}{}{}. If you want to run this job again, change the parameters.

                      (For eg: footballJob executed 2-3 times successfully then it is giving the 2nd error.)

                      Is this always running the same job instance? Even if the footballJob executed 2-3 times, only one entry is there in BATCH_JOB_INSTANCE table.

                      I have one more doubt: how do we execute more than one job using the same QuartzBatchLauncher? Suppose if i want to run footballJob and taskletJob together using the same QuartzBatchLauncher. I added one more entry (<entry key="jobName" value="taskletJob"/>) in jobDetails bean definition of quartz-job-launcher-context.xml. I added
                      final String[] paths = new String[] { "jobs/taskletJob.xml", "jobs/footballJob.xml" };
                      in QuartzBatchLauncher.java
                      But it is trying to execute the taskletJob only and i am getting the error:2 given above.

                      Originally posted by Dave Syer View Post
                      From batch-oracle.properties in the samples project:

                      Code:
                      batch.lob.handler.class=org.springframework.jdbc.support.lob.DefaultLobHandler
                      Did you include that file in your context (the easiest way is by setting a System property environment=oracle)?

                      Actually the lob handler is only there to support very old versions of Oracle, and we'll be removing it from the samples at some point anyway. If you have Oracle 10g or above you don't need it so you can root out the bean definition that is causing problems and delete it.

                      Comment


                      • #12
                        The errors look normal. Read the user guide and follow the JobLauncher Javadocs for explanations. The sample is deliberately artificial because it launches the same instance of the football job repeatedly (which is bound to fail after the first successful execution).

                        I'm not sure what you are doing with the additional jobs. Each one should have its own JobDetailBean and probably its own trigger. Please refer to the Spring reference guide and Javadocs for SchedulerFactoryBean and JobDetailBean for more information.

                        Comment


                        • #13
                          Notification mechanism

                          What are the notification mechanisms provided by spring batch?

                          Originally posted by Dave Syer View Post
                          The errors look normal. Read the user guide and follow the JobLauncher Javadocs for explanations. The sample is deliberately artificial because it launches the same instance of the football job repeatedly (which is bound to fail after the first successful execution).

                          I'm not sure what you are doing with the additional jobs. Each one should have its own JobDetailBean and probably its own trigger. Please refer to the Spring reference guide and Javadocs for SchedulerFactoryBean and JobDetailBean for more information.

                          Comment


                          • #14
                            I don't understand the question. Can you give us a use case?

                            Comment


                            • #15
                              Notification like an e-mail to be sent after the execution of every job.

                              Comment

                              Working...
                              X