Announcement Announcement Module
No announcement yet.
ItemCount and skipcount for HibernateAwareItemWriter Page Title Module
Move Remove Collapse
Conversation Detail Module
  • Filter
  • Time
  • Show
Clear All
new posts

  • ItemCount and skipcount for HibernateAwareItemWriter

    How does itemcount, skipcount and rollbackcount will work in case of using HibernateAwareItemWriter ? Is this different from the normal JDBC scenario?

    I have four parent records and a few children records for each parent.
    Three children for first two parents and one each for remaining two parents.
    Suppose if failure occurs for a child insertion of second parent record(all other records are fine), all other three parent records and its children will be inserted into tables.

    I've commit interval of 3. Using JDBC writer, i got itemcount as 4 and skipcount as 1. This is fine.
    But when i used HibernateAwareItemWriter, i got itemcount as 7 and skipcount as 1. This happens for the first run after starting the server(batch is running on a app server). If i run the same scenario (with out server restart), i am getting the values 4 and 1.

    Would anybody please tell me why this mismatch ?
    Should i add anything more in HibernateAwareItemWriter ?
    My writer def in job.xml looks like this:

    <bean id="hibernateItemWriter"
        <property name="sessionFactory" ref="sessionFactory" />
        <property name="delegate" ref="customDelegateWriter" />
    	<bean id="customDelegateWriter" class="com.DelegateWriter">
    		<property name="delegate" ref="customWriter" />
    	<bean id="customWriter"
    		<property name="childDao" ref="childDao" />

  • #2
    Hi Johny,

    As far as the MISMATCH is concerned, the API documentation of HibernateAwareItemWriter says "the set of failed items is stored in a collection internally, and this collection is never cleared" , i feel this problem can be solved by loading the context every time before running the job as mentioned in the below post. Chudak as further explained even the implementation details very clearly.

    Anyway, i'm not sure why the ItemCount and SkipCount differs when HibernateAwareItemWriter is used. Even i'm facing the same problem.

    May i request senior members like Dave, Lucas, Chudak to shed some light on this.


    • #3
      It is to be expected that the counts do not match. The item count (in 1.x) is the number of items read from the item reader, including resets and re-reads after a rollback. With Hibernate we have to re-play the whole chunk after a failure, item by item, so that we can identify a particular item that failed (if indeed there is one). WIth JDBC you don't have to do that, so the counts will be different.

      By the way, in 2.0 there is a separate itemReadCount that should be more what you probably expect, since we never (by default) re-read the items from the item reader.


      • #4
        Dave, Thanks for replying...

        I wont be able to switch to new version of spring batch now. Moreover i guess stable version of version 2 is not yet released. Hence Can you please suggest if there is any work around to find number of Items Processed and number of Items skipped in spring batch 1.1.2.


        • #5
          I would store them in the ExecutionContext using listeners, or by wrapping your reader/writer.


          • #6
            This is bit interesting... How will the Listener(like ItemWriteListener) identify whether the item is being processed for the first time or it is a re-read due to previous failure of the chunk?

            Can you please let me know which listener can be used to achieve this and how..


            • #7
              You would need to be able to identify the items that have already been processed. Not easy in general, which is why we don't do it in the framework. But you have some business knowledge of the items, so you can store their primary keys or something until they are successfully processed. You can use a varity of mechanisms to clear a buffer on commit: ChunkListener, TransactionSynchronization, or ItemWriter.flush() - the HibernateAwareItemWriter in 1.x is an example.


              • #8
                Can you kindly elaborate a bit on this?


                • #9
                  where is it?

                  okay, i am obviously new here, but i'll forge ahead with another silly question.

                  where is this HibernateAwareItemWriter? the name sounds like something i want to check out, but i downloaded spring-batch 2.0 M3 yesterday and it's not in any of the jars (javadoc, source, class files) in the dist...

                  [email protected]:~/oss/org.springframework.batch.dist-2.0.0.M3$ ls dist/
                  [email protected]:~/oss/org.springframework.batch.dist-2.0.0.M3$ grep -ir hibernateaware dist/
                  [email protected]:~/oss/org.springframework.batch.dist-2.0.0.M3$


                  • #10
                    HibernateAwareItemWriter isn't needed in 2.0 because its role was basically to flush on a chunk boundary, and the ItemWriter is able to do that itself now. If you are using Hibernate you can just call session.flush() at the end of your ItemWriter.write() in 2.0.