Announcement Announcement Module
Collapse
No announcement yet.
how to retry a failed transaction? Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • how to retry a failed transaction?

    I'm porting an application to Google App Engine. They recommend retrying a transaction if it fails (e.g., busy server). Google's documentation says:
    In a system with optimistic concurrency, it is typical for an application to try the transaction again several times before giving up. JDO only performs the transaction once; the application must repeat the transaction, if desired. For example:
    PHP Code:
            for (int i 0NUM_RETRIESi++) {
                
    pm.currentTransaction().begin();

                
    ClubMembers members pm.getObjectById(ClubMembers.class, "k12345");
                
    members.incrementCounterBy(1);

                try {
                    
    pm.currentTransaction().commit();
                    break;

                } catch (
    JDOCanRetryException ex) {
                    if (
    == (NUM_RETRIES 1)) { 
                        throw 
    ex;
                    }
                }
            } 
    I'm using Spring's JdoTransactionManager and an xml file with tx colon advice and aop colon pointcut to specify what runs with a transaction.

    Is there some way I could add retry code using similar logic as above so that when a JDOCanRetryException occurs it re-runs the method (for NUM_RETRIES times)? Where would I add this code; do I need to extend some class or something? I have no clue.

  • #2
    Have a look at this:

    http://static.springsource.org/sprin...just-tx-advice

    I have a transaction retry interceptor working based on this solution. Make sure that the retry interceptor is outside the transaction interceptor, to insure that each retry is in a new transaction.

    Chris

    Comment


    • #3
      Excellent; thanks Chris.

      Comment


      • #4
        Here's what I came up with. Can you find anything wrong with it?

        http://lumpynose.wordpress.com/2009/...-transactions/

        Comment


        • #5
          Quick work, the logging proves you have this working correctly with the retry outside the transaction.

          I think you should consider removing the transactions around the dao's (they will confuse some later maintainer), if you set up your Junits as described in the link below you will be fine:

          http://static.springsource.org/sprin...testcontext-tx

          Is there some reason you have chosen to use XML, rather than annotations to configure transactions:

          http://static.springsource.org/sprin...ve-annotations

          Annotations make it immediately clear to a class maintainer, that transactions are present.

          There is no delay between your retries, I recommend adding an attribute and setter so than you can tune this. I found this discussion about GAE retries, which shows the use of exponential back-off of the retry interval:

          http://stackoverflow.com/questions/1...gle-app-engine

          Only the final exception is logged, so if for example the first exception was different to the second, you would never know. When you log the exception, you have not logged the stack trace (useful for diagnosis of what is really going on); I have never used SLF4J, so this may be a config thing.

          regards,
          Chris

          Comment


          • #6
            Thanks Chris.

            Yes, I am thinking that I could redo the transaction configuration with annotations. I did it that way because I was following an example and it has the advantage that you can see it all in one place. But overall I greatly prefer annotations.

            Absolutely agree about doing some sort of pause between retries but you can't call thread.sleep because the thread class is persona non grata on GAE. I posted a query about this on the GAE-java mailing list and a Ted Stockwell responded that I don't really need to worry about it;
            It is not necessary to pause before retrying because if a transaction fails with a 'RetryException' it is only because some other transaction was committed and that other transaction made some changes that are incompatible with the failed transactions changes.

            So... suppose you kick off 10 transactions at once. At *most* only 9 of those transactions will fail with a RetryException. If you retry those 9 then at *most* 8 will fail, and so on....
            Which doesn't make me feel any better; I'd still like to do some sort of back off.

            I've updated the retryer so that it's hopefully better about handing the zoo of possible exceptions, and it's using Jakarta Commons Lang ExceptionUtils to print the full exception stack trace on each retry;
            PHP Code:
                public Object retry(final ProceedingJoinPoint pjpthrows Throwable {
                    
            this.log.debug("called");

                    
            Throwable exception = new Throwable("oops");

                    
            int retryCount 0;

                    while (
            retryCount++ < this.maxRetries) {
                        try {
                            return (
            pjp.proceed());
                        }
                        catch (final 
            JDOUserException ex) {
                            
            exception ex;

                            break; 
            // fail
                        
            }
                        catch (final 
            JDOCanRetryException ex) {
                            
            exception ex;

                            
            // retry
                        
            }
                        catch (final 
            JDOException ex) {
                            
            exception ex;

                            
            /**
                             * to quote Google's documentation: If any action
                             * fails due to the requested entity group being in
                             * use by another process, JDO throws a
                             * JDODataStoreException or a JDOException, caused by a
                             * java.util.ConcurrentModificationException.
                             */
                            
            if (!(ex.getCause() instanceof ConcurrentModificationException))
                                break; 
            // fail

                            // retry
                        
            }

                        
            this.log.debug("retryCount: {}, exception: {}",
                                
            Integer.valueOf(retryCount),
                                
            ExceptionUtils.getFullStackTrace(exception));
                    }

                    throw (
            exception);
                } 

            Comment


            • #7
              The quote from the GAE guy is interesting, does this mean that a transaction rejected because another user has modified the entity will cause a retry and therefore risk a "lost update" because the retry will overwrite the changes made by the other user?

              Chris

              Comment


              • #8
                I don't know. There are videos of google guys giving presentations at the last google conference, and the one before; they may explain. They're not exactly exciting to watch. The google app engine data store is very different from an sql database; the rules it plays by are not at all the same.

                Comment


                • #9
                  It should be fairly easy to test if 'lost updates' occur by editing the same entity in two sessions in your app.

                  Chris

                  Comment

                  Working...
                  X