Announcement Announcement Module
No announcement yet.
Spring batch - Transaction Management Page Title Module
Move Remove Collapse
Conversation Detail Module
  • Filter
  • Time
  • Show
Clear All
new posts

  • Spring batch - Transaction Management

    I am developing an application with Spring batch 2.0. Data acces Object layer is implemented via Hibernate.

    I have configured TransactionManager(HibernateTransactionManager) in the step level.

    The job contains threee steps.
    Step 1 - Inserts a control information
    Step 2 - Actual data loading
    Step 3 - Validating the data loading.

    My issue is, the data inserted in step1 is required in Step2, but I found that TransactionManger commits the changes only when the job is completed.

    Please guide me. What changes do I need to make, so that the transaction gets commited after each step.


  • #2
    Well, I would be very surprised if it was always the case.

    Can you check that your caller is not within a transaction when the job is launched? If your caller (for instance a unit test with the Spring test infrastructure) has a transaction, then what you are experiencing is normal since Spring Batch defines REQUIRED transaction demarcation (i.e. if a transaction exists, use it; if not, create one).

    One way to avoid this is to have a custom facade in front of the Job launcher with a transaction demarcation NEVER. That way, a caller that is within a transaction won't be able to start the batch. I wouldn't rely on a job launcher that spawns a new thread as transaction managers are capable of detect that and propagate the transaction context in the created thread.


    • #3
      At what point is the transaction created in spring batch?

      I have a step with commit size of lets say 50, internally the item writer implementation calls a home made service (made with hibernate). My goal is to be able to split the transaction size in case of a failure, lets say that it failed inserting 50 transactions then try with 2 chunks of 25 items and so on, until i commit everything except the record that has issues.

      Is that doable?


      • #4
        When the chunk is about to start, a new transaction is created. Say your chunk size is 10, the reader will be called 10 times, the processor will be called 10 times and the writer once with the 10 elements returned by the processor. Then the commit will occur (Spring Batch will also contribute some metadata to the step execution context).

        This is explained in the documentation.

        What you are describing is already provided out-of-the-box by Spring batch but not with the granularity you would expect. When something fails in the writer, it has no idea which item in the list caused the issue so, if the exception is recoverable, it will treat all elements of the chunk again one by one (i.e. with a chunk of size 1).

        In practice, it is better to perform as much as possible in the reader/processor because if something fails there we know exactly what went wrong.

        Last edited by snicoll; Jun 5th, 2011, 04:48 AM.


        • #5
          Thanks for your answer

          I have a doubt about:

          "When something fails in the writer, it has no idea which item in the list caused the issue so, if the exception is recoverable, it will treat all elements of the chunk again one by one (i.e. with a chunk of size 1)"

          So if a no-rollback exception happens spring batch will automatically generate X number of transactions to write 1 item at a time? Where can i read more about this? i think im looking in the wrong place because i dont find much info about it.



          • #6
            Most users don't need to know about the internal details of retry semantics, so the user guide doesn't cover it in much detail - you have to go to the source code (FaultTolerantChunkProcessor). If you want more you can look at the upcoming books (Spring Batch in Action definitely covers it), or make a JIRA request for the user guide to be updated (community contributions gratefully accepted).


            • #7
              Thanks i will look the FaultTolreantChunkProcessor