Announcement Announcement Module
Collapse
No announcement yet.
Retry test with HibernateCursorItemReader Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • Retry test with HibernateCursorItemReader

    Hi, I'm a newbie with Spring Batch and am using RC1. I encountered a problem when testing 'retry' with HibernateCursorItemReader.

    Here's my case:

    read 5 records from db using HibernateCursorItemReader as item reader
    set retryLimit to 3
    item writer simply throws exception when processing each item

    from my understanding about 'retry', the result will be:

    process 1st record, exception encountered
    retry on 1st record for 2nd time, exception encountered
    retry on 1st record for 3rd time, exception encountered
    ignore 1st record, process 2nd record, exception encountered
    retry on 2nd record for 2nd time, exception encountered
    retry on 2nd record for 3rd time, exception encountered
    ........

    but the actual result is a dead loop of processing the 1st item!

    But when I test RetrySampleFunctionalTests with the same item writer logic (throw exception no matter how), the result was as expected.

    Then I did some debugging on my failed case and found that in SimpleRetryPolicy, method canRetry(RetryContext context), parm 't' is always null and the retry count in context is always 0, hence this method always returns true which results in no-ending for max retry.

    Anybody can tell me what's wrong with it? Or how I suppose to test 'retry' mechanism with HibernateCursorItemReader ?

  • #2
    I would expect a failure in the item writer to cause a rollback, so the first item is re-presented in a new transaction. That's what you are seeing, but the retry never terminates?

    Maybe you haven't configured the retry correctly - could you tell us a bit more about what you are doing and how? Which version of Spring Batch are you using? I recommend you get last night's snapshot if you continue to experience problems (there were some significant changes to the stateful retry stuff yesterday). In fact I am still working on it, so there might be bugs (I'm using BATCH-498 and BATCH-486 to track it).

    Are you using the StatefulRetryStepFactoryBean to set up the step (I assume that's what you mean by "retryLimit", but just checking)?

    Comment


    • #3
      Sorry that I was not in office during the weekend so I could not check my codes. Here's my reply:

      Originally posted by Dave Syer View Post
      I would expect a failure in the item writer to cause a rollback, so the first item is re-presented in a new transaction. That's what you are seeing, but the retry never terminates?
      It seems that the exception thrown from item writer was lost because I could not find it when debugging (see my origin post). And yes, the retry neven ended if I throw exception on each item processed by item writer. And the looping is always on the first item!

      Originally posted by Dave Syer View Post
      Maybe you haven't configured the retry correctly - could you tell us a bit more about what you are doing and how? Which version of Spring Batch are you using? I recommend you get last night's snapshot if you continue to experience problems (there were some significant changes to the stateful retry stuff yesterday). In fact I am still working on it, so there might be bugs (I'm using BATCH-498 and BATCH-486 to track it).
      I'm not sure about the version i am using....where can i see it? I just know I'm using 1.0.0.RC1 and it was downloded 2 weeks ago...and yes I would have a try on the updated version.

      Originally posted by Dave Syer View Post
      Are you using the StatefulRetryStepFactoryBean to set up the step (I assume that's what you mean by "retryLimit", but just checking)?
      Yes i am using the exact bean you mentioned. And here's the detail configuration of the job.

      <bean id="retrySample" parent="simpleJob">
      <property name="steps">
      <bean id="retrySampleStep" parent="simpleStep"
      class="org.springframework.batch.core.step.item.St atefulRetryStepFactoryBean">
      <property name="itemReader" ref="tradeExecutionItemReader" />
      <property name="itemWriter" ref="retrySampleItemWriter" />
      <property name="retryLimit" value="3" />
      <property name="retryableExceptionClasses" value="java.lang.Exception" />
      </bean>
      </property>
      </bean>

      Comment


      • #4
        Can you try a more recent build (e.g. download directly from http://s3browse.com/explore/maven.sp...amework/batch/, or use 1.0.0.FINAL-SNAPSHOT or 1.0.1.dev-SNAPSHOT in Maven)?

        If that is still causing problems we can look into it in more detail, but it shouldn't matter which item reader you use.

        (Please use [code][/code] tags to post code and stack traces.)

        Comment


        • #5
          Dave, I've tried 1.0.0.FINAL-SNAPSHOT and the problem remains. Do you want further information about my test case?

          Comment


          • #6
            Yes please. If you can package it up as a Maven project that is ideal, otherwise I will take a look at whatever you can provide. Don't try and upload an attachment with 50MB of jars in it (there's a limit I think anyway).

            Comment


            • #7
              Originally posted by Dave Syer View Post
              Yes please. If you can package it up as a Maven project that is ideal, otherwise I will take a look at whatever you can provide. Don't try and upload an attachment with 50MB of jars in it (there's a limit I think anyway).
              Here I attached the project interchange zip file (I'm using IBM RAD7.0) which contains my test classes & config files.

              The testcase class is test/java/RetryJobTest.java and config file is in resources/jobs/rtp/rtpBatchJobs.xml

              Please notice that you need to configure the queryStirng in your own hibernate item reader and also use some new domain objects and re-write item writers if you want to run the case....otherwise please just take a look at the codes

              Comment


              • #8
                Thanks for that, it was very helpful. It is easy to explain why you had a problem, but I will open a ticket in JIRA to try and make the failure more obvious. There are a couple of sections in the reference guide that cover this, but I can see it is going to be tough for people, so we'll try and make things easier. Section 6.2.2.1 is th only one in the currently deployed version of the manual, but I know there are other people with this issue, so we'll cover it in more depth somewhere else as well.

                You need to provide a better key generator for your items. Either supply an ItemKeyGenerator to the step factory bean, or use the default and implement equals() and hashCode() in the TradeExecution object (e.g. based on the primary key in your case would be fine, since you only read the objects, and the primary key is never null).

                Comment


                • #9
                  http://jira.springframework.org/browse/BATCH-537. Your original example would throw an exception after a (configurable) number of attempts now - just counting the number of items in the retry cache and barfing when it reaches a limit. There are also some checks to see that the hashCode of failed items doesn't change on write, and a lot of new Javadocs and exception messages that hopefully help a lot.

                  Comment

                  Working...
                  X