Announcement Announcement Module
Collapse
No announcement yet.
HibernateItemWriter lazy loading problem Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • HibernateItemWriter lazy loading problem

    I've setup a hibernate job based on the hibernateJob.xml example, with one difference - I'm trying to use lazy loading.

    Setup:

    Hibernate Entities:
    A----*B
    The hibernate one-to-many relationship between A and B is defined to be lazy


    hibernateItemReader:
    <property name="queryString" value="from A" />
    <property name="useStatelessSession" value="false"/>


    hiberateWriter:
    public void write( Object data ) throws Exception
    {
    A a = (A) data;
    a.addB( new B() );
    a.addB( new B() );
    getSession().saveOrUpdate( a );
    }


    1st issue:
    If the hibernateItemReader does not return any Items (ie. the hibernate query does not return any results) then I get the following exception reported when ever I run the job, although the job appears to return sucessfully:

    org.hibernate.SessionException: Session is closed!
    at org.hibernate.impl.AbstractSessionImpl.errorIfClos ed(AbstractSessionImpl.java:49)
    at org.hibernate.impl.SessionImpl.getBatcher(SessionI mpl.java:260)
    at org.hibernate.impl.AbstractScrollableResults.close (AbstractScrollableResults.java:99)
    at org.springframework.batch.io.cursor.HibernateCurso rItemReader.close(HibernateCursorItemReader.java:1 12)
    at org.springframework.batch.io.cursor.HibernateCurso rItemReader.destroy(HibernateCursorItemReader.java :152)
    at org.springframework.beans.factory.support.Disposab leBeanAdapter.destroy(DisposableBeanAdapter.java:1 54):


    2nd issue:
    If the hibernateItemReader does return results I get the following exception in the hibernateWriter when getSession().saveOrUpdate( a ) is called:

    org.hibernate.HibernateException: Illegal attempt to associate a collection with two open sessions
    at org.hibernate.collection.AbstractPersistentCollect ion.setCurrentSession(AbstractPersistentCollection .java:410)
    at org.hibernate.event.def.OnUpdateVisitor.processCol lection(OnUpdateVisitor.java:43)


    Any ideas ?

  • #2
    Does it solve your problem if you turn off lazy loading? Just want to make sure the root cause is correct (no idea what's wrong yet).

    Comment


    • #3
      If I disable lazy loading, I still get the same exception. Should have checked that!... so its not as I assumed a lazy loading problem.

      A bit more info... If I change the hibernateWriter to :

      public void write( Object data ) throws Exception
      {
      A a = (A) getSession().get( A.class , ((A)data).getId() );
      a.addB( new B() );
      a.addB( new B() );
      getSession().saveOrUpdate( a );
      }


      Then this works fine.

      Comment


      • #4
        I think the the point here is that you have two hibernate sessions: one for reader and another for writer. So:

        1st: when there are no items the writer is never called and does not have a chance to open it's session
        UPDATE: sorry this is wrong, it's the reader that throws exception on close, not writer
        UPDATE2: the error is not specific to your scenario, the original hibernateJob behaves the same way - close is called twice
        http://jira.springframework.org/browse/BATCH-343

        2nd: the item is associated with the reader's session and you're trying to associate to writer's session

        I guess that explains why you're having trouble, now to figure out a solution...
        Last edited by robert.kasanicky; Feb 6th, 2008, 07:32 AM.

        Comment


        • #5
          I've found another issue:

          Using the hibernateWriter:

          public void write( Object data ) throws Exception
          {
          A a = (A) getSession().get( A.class , ((A)data).getId() );
          a.addB( new B() );
          a.addB( new B() );
          getSession().saveOrUpdate( a );
          }


          If I attempt to run the hibernateJob on a Weblogic server using a Weblogic datasource and the Weblogic JTA transaction manager I get the following exception after the chunk commit is called:

          Caused by: java.sql.SQLException: Result set already closed
          at weblogic.jdbc.wrapper.ResultSet.checkResultSet(Res ultSet.java:102)
          at weblogic.jdbc.wrapper.ResultSet.preInvocationHandl er(ResultSet.java:58)
          at weblogic.jdbc.wrapper.ResultSet_oracle_jdbc_driver _ScrollableResultSet.beforeFirst()V(Unknown Source)
          at org.hibernate.impl.ScrollableResultsImpl.beforeFir st(ScrollableResultsImpl.java:151)
          at org.springframework.batch.io.cursor.HibernateCurso rItemReader.reset(HibernateCursorItemReader.java:2 48)



          What seems to be happening is that when the transaction is commited (by the JTA Transaction Manager), the ScrollableResultSet cursor automatically gets closed, and thus when the next item is read I get the above error.

          If I attempt to Isolate the HibernateCursorItemReader transaction then everything works fine:


          <bean id="transactionManager" class="org.springframework.transaction.jta.WebLogi cJtaTransactionManager">
          <property name="transactionManagerName" value="javax.transaction.TransactionManager" />
          </bean>

          <bean id="propagationNotSupportedTransactionProxy" class="org.springframework.transaction.interceptor .TransactionProxyFactoryBean" abstract="true">
          <property name="transactionManager">
          <ref bean="transactionManager"/>
          </property>
          <property name="transactionAttributes">
          <props>
          <prop key="*">PROPAGATION_NOT_SUPPORTED</prop>
          </props>
          </property>
          </bean>

          <bean id="hibernateItemReader" parent="propagationNotSupportedTransactionProxy">
          <property name="target">
          <bean class="org.springframework.batch.io.cursor.Hiberna teCursorItemReader" scope="step">
          <aop:scoped-proxy />
          <property name="queryString" value="from A" />
          <property name="sessionFactory" ref="sessionFactory" />
          <property name="useStatelessSession" value="false"/>
          </bean>
          </property>
          </bean>


          Is this the correct way to get around the problem ?

          Comment


          • #6
            Can you please paste the whole stacktrace? The last line of it says the hibernate reader is being reset which indicates some error was encountered before.

            Comment


            • #7
              - Job: [SimpleJob: [name=hibernateJob]] launched with the following parameters: [{}{}{}]
              - select a0_.id as id0_, a0_.attrib_1 as attrib2_0_, a0_.attrib_2 as attrib3_0_ from a a0_
              - select a0_.id as id0_0_, a0_.attrib_1 as attrib2_0_0_, a0_.attrib_2 as attrib3_0_0_ from a a0_ where a0_.id=?
              - select a0_.id as id0_0_, a0_.attrib_1 as attrib2_0_0_, a0_.attrib_2 as attrib3_0_0_ from a a0_ where a0_.id=?
              - select a0_.id as id0_0_, a0_.attrib_1 as attrib2_0_0_, a0_.attrib_2 as attrib3_0_0_ from a a0_ where a0_.id=?
              - select hibernate_sequence.nextval from dual
              - select hibernate_sequence.nextval from dual
              - select hibernate_sequence.nextval from dual
              - select hibernate_sequence.nextval from dual
              - select hibernate_sequence.nextval from dual
              - select hibernate_sequence.nextval from dual
              - insert into b (attrib_1, my_index, a_id, id) values (?, ?, ?, ?)
              - insert into b (attrib_1, my_index, a_id, id) values (?, ?, ?, ?)
              - insert into b (attrib_1, my_index, a_id, id) values (?, ?, ?, ?)
              - insert into b (attrib_1, my_index, a_id, id) values (?, ?, ?, ?)
              - insert into b (attrib_1, my_index, a_id, id) values (?, ?, ?, ?)
              - insert into b (attrib_1, my_index, a_id, id) values (?, ?, ?, ?)
              - update a set attrib_1=?, attrib_2=? where id=?
              - SQL Error: 0, SQLState: null
              - Result set already closed
              - TransactionSynchronization.afterCompletion threw exception
              org.hibernate.exception.GenericJDBCException: exception calling beforeFirst()
              at org.hibernate.exception.SQLStateConverter.handledN onSpecificException(SQLStateConverter.java:103)
              at org.hibernate.exception.SQLStateConverter.convert( SQLStateConverter.java:91)
              at org.hibernate.exception.JDBCExceptionHelper.conver t(JDBCExceptionHelper.java:43)
              at org.hibernate.exception.JDBCExceptionHelper.conver t(JDBCExceptionHelper.java:29)
              at org.hibernate.impl.ScrollableResultsImpl.beforeFir st(ScrollableResultsImpl.java:154)
              at org.springframework.batch.io.cursor.HibernateCurso rItemReader.reset(HibernateCursorItemReader.java:2 48)
              at org.springframework.batch.item.stream.SimpleStream Manager$3$2.execute(SimpleStreamManager.java:189)
              at org.springframework.batch.item.stream.SimpleStream Manager.iterate(SimpleStreamManager.java:212)
              at org.springframework.batch.item.stream.SimpleStream Manager.access$100(SimpleStreamManager.java:44)
              at org.springframework.batch.item.stream.SimpleStream Manager$3.afterCompletion(SimpleStreamManager.java :186)
              at org.springframework.transaction.support.Transactio nSynchronizationUtils.invokeAfterCompletion(Transa ctionSynchronizationUtils.java:133)
              at org.springframework.transaction.support.AbstractPl atformTransactionManager.invokeAfterCompletion(Abs tractPlatformTransactionManager.java:951)
              at org.springframework.transaction.support.AbstractPl atformTransactionManager.triggerAfterCompletion(Ab stractPlatformTransactionManager.java:926)
              at org.springframework.transaction.support.AbstractPl atformTransactionManager.processRollback(AbstractP latformTransactionManager.java:829)
              at org.springframework.transaction.support.AbstractPl atformTransactionManager.rollback(AbstractPlatform TransactionManager.java:777)
              at org.springframework.batch.item.stream.SimpleStream Manager.rollback(SimpleStreamManager.java:229)
              at org.springframework.batch.execution.step.simple.Si mpleStepExecutor$1.doInIteration(SimpleStepExecuto r.java:245)
              at org.springframework.batch.repeat.support.RepeatTem plate.getNextResult(RepeatTemplate.java:324)
              at org.springframework.batch.repeat.support.RepeatTem plate.executeInternal(RepeatTemplate.java:201)
              at org.springframework.batch.repeat.support.RepeatTem plate.iterate(RepeatTemplate.java:131)
              at org.springframework.batch.execution.step.simple.Si mpleStepExecutor.execute(SimpleStepExecutor.java:1 86)
              at org.springframework.batch.execution.step.simple.Re peatOperationsStep.execute(RepeatOperationsStep.ja va:89)
              Caused by: java.sql.SQLException: Result set already closed
              at weblogic.jdbc.wrapper.ResultSet.checkResultSet(Res ultSet.java:102)
              at weblogic.jdbc.wrapper.ResultSet.preInvocationHandl er(ResultSet.java:58)
              at weblogic.jdbc.wrapper.ResultSet_oracle_jdbc_driver _ScrollableResultSet.beforeFirst()V(Unknown Source)
              at org.hibernate.impl.ScrollableResultsImpl.beforeFir st(ScrollableResultsImpl.java:151)
              at org.springframework.batch.io.cursor.HibernateCurso rItemReader.reset(HibernateCursorItemReader.java:2 48)
              at org.springframework.batch.item.stream.SimpleStream Manager$3$2.execute(SimpleStreamManager.java:189)
              at org.springframework.batch.item.stream.SimpleStream Manager.iterate(SimpleStreamManager.java:212)
              at org.springframework.batch.item.stream.SimpleStream Manager.access$100(SimpleStreamManager.java:44)
              at org.springframework.batch.item.stream.SimpleStream Manager$3.afterCompletion(SimpleStreamManager.java :186)
              at org.springframework.transaction.support.Transactio nSynchronizationUtils.invokeAfterCompletion(Transa ctionSynchronizationUtils.java:133)
              at org.springframework.transaction.support.AbstractPl atformTransactionManager.invokeAfterCompletion(Abs tractPlatformTransactionManager.java:951)
              at org.springframework.transaction.support.AbstractPl atformTransactionManager.triggerAfterCompletion(Ab stractPlatformTransactionManager.java:926)
              at org.springframework.transaction.support.AbstractPl atformTransactionManager.processRollback(AbstractP latformTransactionManager.java:829)
              at org.springframework.transaction.support.AbstractPl atformTransactionManager.rollback(AbstractPlatform TransactionManager.java:777)
              at org.springframework.batch.item.stream.SimpleStream Manager.rollback(SimpleStreamManager.java:229)
              at org.springframework.batch.execution.step.simple.Si mpleStepExecutor$1.doInIteration(SimpleStepExecuto r.java:245)
              at org.springframework.batch.repeat.support.RepeatTem plate.getNextResult(RepeatTemplate.java:324)
              at org.springframework.batch.repeat.support.RepeatTem plate.executeInternal(RepeatTemplate.java:201)
              at org.springframework.batch.repeat.support.RepeatTem plate.iterate(RepeatTemplate.java:131)
              at org.springframework.batch.execution.step.simple.Si mpleStepExecutor.execute(SimpleStepExecutor.java:1 86)
              at org.springframework.batch.execution.step.simple.Re peatOperationsStep.execute(RepeatOperationsStep.ja va:89)
              at org.springframework.batch.execution.job.simple.Sim pleJob.execute(SimpleJob.java:82)
              at org.springframework.batch.execution.launch.SimpleJ obLauncher$1.run(SimpleJobLauncher.java:85)
              at org.springframework.core.task.SyncTaskExecutor.exe cute(SyncTaskExecutor.java:49)
              at org.springframework.batch.execution.launch.SimpleJ obLauncher.run(SimpleJobLauncher.java:80)
              at org.springframework.batch.execution.bootstrap.supp ort.SimpleExportedJobLauncher.run(SimpleExportedJo bLauncher.java:159)
              at org.springframework.batch.execution.bootstrap.supp ort.SimpleExportedJobLauncher.run(SimpleExportedJo bLauncher.java:134)
              at jrockit.reflect.VirtualNativeMethodInvoker.invoke( Ljava.lang.Object;[Ljava.lang.ObjectLjava.lang.Object;(Unknown Source)
              at java.lang.reflect.Method.invoke(Ljava.lang.Object;[Ljava.lang.Object;I)Ljava.lang.Object;(Unknown Source)
              - SQL Error: 0, SQLState: null
              - Result set already closed
              - SQL Error: 0, SQLState: null
              - Result set already closed

              Comment


              • #8
                Frankly it is not clear to me why the cursor is being closed (Lucas?, Dave?), but I see nothing wrong with your solution. The reader does not depend on any transactional magic - it is told by the framework all it needs to know, so things should just work.

                Btw. please use code tags when pasting stacktraces

                Comment


                • #9
                  We wrote 2 components called HibernateInputSource and HibernateBatchingObjectPersister, functionalitywise they are very similar if not equal to HibernateCursorItemReader and HibernateAwareItemWriter. (We are using a snapshot version of spring-batch from last september when those components were not yet in the framework so we were forced to write something ourselves.)

                  What's more we have struggled for days if not weeks with these very exceptions as well: Illegal attempt to associate a collection with two open sessions, Session closed, Result set already closed.

                  1) Robert rightly points out it is crucial that you're running both the reader and the writer in the same hibernatesession.

                  2) However, looking at the sources of HibernateCursorItemReader and HibernateAwareItemWriter this seems impossible to accomplish at the moment. The HibernateCursorItemReader creates its own session through the hibernate API: sessionFactory.openSession(). The reader on the other hand uses the spring API to create another one.

                  => Here is how we "solved" this issue:

                  + Our HibernateInputSource calls sessionFactory.currentSession() instead

                  + We set the hibernate.current_session_context_class attribute to "managed". If you don't do this the session will be closed each time a transaction is committed (everytime the commit-interval configured on the step is reached)

                  + We define a bean in the applicationcontext with an init-method in which we set the currentsessioncontext through ManagedSessionContext.bind(getSession()) and a destroy-method in which we remove it through ManagedSessionContext.unbind(sessionFactory)

                  + In these 2 methods we also influence the currentsessionmechanism of spring through TransactionSynchronizationManager.bindResource(ses sionFactory, new SessionHolder(getSession())) and TransactionSynchronizationManager.unbindResource(s essionFactory). If you don't do this the transaction settings only apply to spring-batch and not to the HibernateInputSource or the HibernateBatchingObjectPersister. Meaning everything will be commited directly dispite the commitinterval being set to something else.


                  3) After we did all this the solution became more or less usable, unfortunately there is still 1 thing which we never solved. Due to time constraints we had to work around it by closing and opening the cursor each time the commit interval is reached. Anyhow, the problem is that after a transaction is commited the cursor thinks it is still open, however, you can not read from it anymore because Spring seems to have closed the underlying database connection. This gives the famous "Result set already closed" exception you are now experiencing.

                  Comment


                  • #10
                    If you use the HibernateCursorItemReader you have to treat the objects you get from it with care because they *do* come from a different session, intentionally (so the cursor can stay open for the duration of the step). By default it uses a StatelessSession, so you explicitly get *detached* objects from it. I thought that was pretty sensible, but some users wanted to do lazy loading with those objects so we provided the useStatelessSession flag as an option. Once you set that to "false" it is your responsibility to deal with the consequences. Can anyone see any way for the framework to help out here?

                    Comment


                    • #11
                      Originally posted by Dave Syer View Post
                      If you use the HibernateCursorItemReader you have to treat the objects you get from it with care because they *do* come from a different session, intentionally
                      Dave, if this is done intentionally then fine, but I would really document this very well because it's not just about being carefull what you do, it's actually quite dangerous if you do anything at all with the object apart from reading its properties. If you change the object Hibernate will pick this up and write back the changes to the database. This is probably what you'd want, but because it's running in another session/transaction the commit interval is completely ignored and everything you do is written to the DB directly, obviously this means that a rollback can never work!

                      Originally posted by Dave Syer View Post
                      (so the cursor can stay open for the duration of the step).
                      On most DB drivers you can set the property resultsetholdability which will also allow the cursor to be open across multiple transactions.

                      Originally posted by Dave Syer View Post
                      By default it uses a StatelessSession, so you explicitly get *detached* objects from it. I thought that was pretty sensible, but some users wanted to do lazy loading with those objects so we provided the useStatelessSession flag as an option. Once you set that to "false" it is your responsibility to deal with the consequences. Can anyone see any way for the framework to help out here?
                      I was probably one of those users. :o
                      I'm not sure what the framework can do, but I'd certainly like to point out that the way it is set up now the HibernateCursorItemReader in stateful mode can not be used for its most common use case, modifying data through Hibernate alone. It can only be used in combination with another persistance mechanism.
                      • As already explained above using Hibernate's writeback is not an option because all changes will be executed outside of the transactions
                      • But using a HibernateAwareItemWriter to persist the changes is not an option either. All objects retrieved through the HibernateCursorItemReader will be associated with the session meaning we can't save them in the HibernateAwareItemWriter. (As an object can never belong to 2 hibernatesessions)

                      Anyhow, as already mentioned in one of my previous posts we have forced the 2 to use the same session and to run in the transaction started by spring batch and it seems to be working fine apart from the fact the Spring currentSession mechanism is closing the underlying DB connection of the cursor after each transaction forcing us to close and reopen the cursor every time. Obviously we should address this in the future as it might cause rows being processed twice or not at all if other users are inserting or deleting rows. This is potentialy already the case if you restart a failed job (Because the restart data is a simple counter) so for now we can live with that.
                      Last edited by sm@h; Feb 8th, 2008, 04:50 AM.

                      Comment


                      • #12
                        sm@h, Dave, it looks like I have finally hit the same problem. Perhaps I will not say something new to you, but to somehow define the possible solution I would like to pin-point few things:
                        • StatelessSession, used by e.g. HibernateCursorItemReader is absolutely fine for certain applications, but (from JavaDoc):
                          • cascaded operations to associated instances are not performed by a stateless session
                          • collections are ignored by a stateless session
                          so if one plans to use something from above then seek for other approach
                        • if one is using the stateful (normal) Session for the entities which have lazy associations, he needs to extend the Hibernate session lifespan for the time while all items are read (see the example of MyItemReader below); this new session A will be associated with no transaction (e.g. only read-only operations are allowed)
                        • that automatically means the ItemWriter can benefit from lazy loading, but if it wants to modify the item and write it back – it fails, as the lazy collection is associated with another session A, while ItemWriter is using session B, which is automatically created together with new transaction started by TaskletStep#doExecute()
                        • in order to make ItemWriter to pickup the same Hibernate session which was opened by ItemReader, one needs to open a transaction before ItemReader opens a Hibernate session
                        • HibernateTransactionManager will not close the Hibernate session automatically on ItemWriter commit (it uses SessionFactoryUtils.closeSessionOrRegisterDeferred Close()) if it was marked for deferred closing

                        Above said might not be correct (please, correct me when I am wrong) and perhaps there is no workaround. My vision to the solution is to allow somehow the transaction to be created for the complete step, like here:

                        Code:
                        T1 {
                         // Beginning of the step
                         - Open streams
                        
                         chunk loop in T2 { // TaskletStep.ChunkTransactionCallback#doInTransaction()
                           - Read items
                           - Write items
                           - T2.commit()
                         }
                        
                         - Close streams
                        }
                        In our case we will leave transactional attributes for T2 as default, that means the T2 is inherited from T1 (basically, T2 == T1). In case REQUIRES_NEW is specified for T2, new Hibernate session will be created (and things will break).

                        I see the light at the end of the tunnel, but unexpectedly the problem arised with ChunkTransactionCallback, as for T2 (which joins T1) transaction synchronization is switched off (DefaultTransactionStatus.isNewSynchronization() is false and ChunkTransactionCallback#afterCompletion() is not invoked, so the semaphore is not released, which causes the second data chunk to hang). I tried to guess, why this is default Spring behaviour to skip transaction synchronization for joined transaction...

                        If one can help or advise here, I would be thankful.

                        Code:
                        public class MyItemReader<T> extends ItemStreamSupport implements ItemReader<T> {
                        
                        	@Autowired
                        	private SessionFactory sessionFactory;
                        
                        	@Autowired
                        	private PlatformTransactionManager transactionManager;
                        
                        	private MyIterator<T> cursor;
                        
                        	private TransactionStatus transactionStatus;
                        
                        	@Override
                        	public IndexedDocument read() {
                        		if (!cursor.hasNext()) {
                        			return null;
                        		}
                        
                        		return cursor.next();
                        	}
                        
                        	@Override
                        	public void open(ExecutionContext executionContext) {
                        		SessionFactoryUtils.initDeferredClose(sessionFactory);
                        
                        		final DefaultTransactionDefinition transactionDefinition = new DefaultTransactionDefinition();
                        		transactionDefinition.setPropagationBehavior(TransactionDefinition.PROPAGATION_REQUIRED);
                        
                        		transactionStatus = transactionManager.getTransaction(transactionDefinition);
                        
                        		cursor = ... open a cursor (e.g. a Query) ...;
                        	}
                        
                        	@Override
                        	public void close() throws ItemStreamException {
                        		if (!transactionStatus.isCompleted()) {
                        			// This situation is not normal, as transaction should have been committed by job step:
                        			transactionManager.commit(transactionStatus);
                        		}
                        
                        		cursor.close();
                        		cursor = null;
                        
                        		SessionFactoryUtils.processDeferredClose(sessionFactory);
                        	}
                        }
                        Last edited by dma_k; Mar 10th, 2011, 05:51 PM.

                        Comment


                        • #13
                          The best advice I can offer is not to use Hibernate or JPA for your ItemReader. If you must do, then do not use the items' lazy-loaded associations downstream. The experiments you are making with TX boundaries we would not want to support because you would then not be able to process large data sets.

                          Comment


                          • #14
                            Originally posted by Dave Syer View Post
                            The best advice I can offer is not to use Hibernate or JPA for your ItemReader. If you must do, then do not use the items' lazy-loaded associations downstream.
                            The problem also rises with non-lazy one-to-one and one-to-many associations:
                            • one can't use StatelessSession to load such entities
                            • using StatefulSession automatically raises exception for HibernateProxy's, created for such associations, saying that proxy cannot be associated with two concurrent sessions
                            At the moment I need to somehow evict the Hibernate entity from the cache of the original Hibernate session:

                            PHP Code:
                            import org.hibernate.Session;
                            import org.hibernate.proxy.HibernateProxy;

                            if (
                            entity.getSomeProperty() instanceof HibernateProxy) {
                                
                            HibernateProxy proxy = (HibernateProxyentity.getSomeProperty();
                                ((
                            Sessionproxy.getHibernateLazyInitializer().getSession()).evict(entity);
                                
                            proxy.getHibernateLazyInitializer().unsetSession();

                            The experiments you are making with TX boundaries we would not want to support because you would then not be able to process large data sets.
                            I see the way out if Spring supports multiple commits per (Hibernate) session. This does not break Spring Batch policy to commit after each chunk. However I have not found any way to do it "legally" because after a transaction is committed, it can't be continued. I've created a new post for Spring Data with my ideas; I hope somebody can give me a hint or tell me whether I have ideologically incorrect approach.

                            Comment


                            • #15
                              After trying different approaches, I've come to the following.

                              First, general considerations:
                              • It would be nice if HibernateCursorItemReader provides a method to access initialized flag, e.g. public isInitialized() { return initialized; }
                              • It would be nice if HibernateCursorItemReader provides a way to access helper (make the field protected or add protected setter/getter), e.g. protected setHelper(HibernateItemReaderHelper helper) { this.helper = helper; }

                              Well, the first approach is to enable "deferred" opening of the session. In this case the newly opened session will join the existing transaction. It is impossible to implement this method on the basis of the existing Batch classes, as HibernateItemReaderHelper uses sessionFactory.openSession() directly instead of SessionFactoryUtils.getSession(sessionFactory, true).

                              Code:
                              /**
                               * This class defers opening of Hibernate session just before the first items is read. It relays on the fact, that
                               * opening of the stream occurs outside the transaction and reading -- inside the transaction.<br>
                               * Note: this approach is the only one in case your entities have lazily loading properties (e.g. collections).
                               */
                              public class DeferredHibernateCursorItemReader<T> extends ItemStreamSupport implements ItemReader<T> {
                              
                              	private boolean			initialized	= false;
                              
                              	private SessionFactory		sessionFactory;
                              
                              	private Session			session;
                              
                              	private String			queryName;
                              
                              	private ScrollableResults	cursor;
                              
                              	public void setSessionFactory(SessionFactory sessionFactory) {
                              		this.sessionFactory = sessionFactory;
                              	}
                              
                              	public void setQueryName(String queryName) {
                              		this.queryName = queryName;
                              	}
                              
                              	/**
                              	 * @see org.springframework.batch.item.ItemReader#read()
                              	 */
                              	public T read() throws Exception, UnexpectedInputException, ParseException, NonTransientResourceException {
                              		if (!initialized) {
                              			session = SessionFactoryUtils.getSession(sessionFactory, true);
                              			initialized = true;
                              			cursor = session.getNamedQuery(queryName).scroll(ScrollMode.FORWARD_ONLY);
                              		}
                              
                              		if (cursor.next()) {
                              			Object[] data = cursor.get();
                              
                              			@SuppressWarnings("unchecked")
                              			T item = (T) data[0];
                              			return item;
                              		}
                              
                              		return null;
                              	}
                              
                              	/**
                              	 * @see org.springframework.batch.item.ItemStream#close()
                              	 */
                              	@Override
                              	public void close() throws ItemStreamException {
                              		SessionFactoryUtils.releaseSession(session, sessionFactory);
                              	}
                              }
                              Another difficulty is in HibernateItemWriter, which checks that entity is not associated with this session before saving it (why?): hibernateTemplate.contains(item). So when the item is updated unconditionally, it everything works as intended:

                              Code:
                              public class HibernateItemWriter<T> extends org.springframework.batch.item.database.HibernateItemWriter<T> {
                              
                              	/**
                              	 * @see org.springframework.batch.item.database.HibernateItemWriter#doWrite(org.springframework.orm.hibernate3.HibernateOperations,
                              	 *      java.util.List)
                              	 */
                              	@Override
                              	protected void doWrite(HibernateOperations hibernateTemplate, List<? extends T> items) {
                              		for (T item : items) {
                              			hibernateTemplate.saveOrUpdate(item);
                              		}
                              	}
                              }
                              Another approach is to initialize all lazy collections before returning the entity. It is more friendly in sense of transaction separation, but requires more code:

                              Code:
                              import org.apache.commons.collections.IteratorUtils;
                              import org.hibernate.ScrollMode;
                              import org.hibernate.ScrollableResults;
                              import org.hibernate.Session;
                              import org.hibernate.SessionFactory;
                              import org.hibernate.collection.PersistentCollection;
                              import org.hibernate.engine.SessionImplementor;
                              import org.hibernate.engine.StatefulPersistenceContext;
                              import org.hibernate.proxy.HibernateProxy;
                              import org.hibernate.util.IdentityMap;
                              import org.springframework.batch.item.ExecutionContext;
                              import org.springframework.batch.item.ItemReader;
                              import org.springframework.batch.item.ItemStreamException;
                              import org.springframework.batch.item.ItemStreamSupport;
                              import org.springframework.batch.item.NonTransientResourceException;
                              import org.springframework.batch.item.ParseException;
                              import org.springframework.batch.item.UnexpectedInputException;
                              import org.springframework.orm.hibernate3.SessionFactoryUtils;
                              
                              /**
                               * This class clears the Hibernate persistence context after fetching the entity.<br>
                               * Note: All lazy collections are loaded. This will assure that the entity is usable in detached state.
                               */
                              public class InitializingHibernateCursorItemReader<T> extends ItemStreamSupport implements ItemReader<T> {
                              
                              	private SessionFactory		sessionFactory;
                              
                              	private Session			session;
                              
                              	private boolean			initialized;
                              
                              	private String			queryName;
                              
                              	private ScrollableResults	cursor;
                              
                              	public void setSessionFactory(SessionFactory sessionFactory) {
                              		this.sessionFactory = sessionFactory;
                              	}
                              
                              	public void setQueryName(String queryName) {
                              		this.queryName = queryName;
                              	}
                              
                              	/**
                              	 * @see org.springframework.batch.item.ItemStreamSupport#open(org.springframework.batch.item.ExecutionContext)
                              	 */
                              	@Override
                              	public void open(ExecutionContext executionContext) throws ItemStreamException {
                              		if (!initialized) {
                              			session = SessionFactoryUtils.getSession(sessionFactory, true);
                              			cursor = session.getNamedQuery(queryName).scroll(ScrollMode.FORWARD_ONLY);
                              			initialized = true;
                              		}
                              	}
                              
                              	/**
                              	 * @see org.springframework.batch.item.ItemStream#close()
                              	 */
                              	@Override
                              	public void close() throws ItemStreamException {
                              		SessionFactoryUtils.releaseSession(session, sessionFactory);
                              	}
                              
                              	/**
                              	 * @see org.springframework.batch.item.ItemReader#read()
                              	 */
                              	public T read() throws Exception, UnexpectedInputException, ParseException, NonTransientResourceException {
                              		if (cursor.next()) {
                              			Object[] data = cursor.get();
                              
                              			@SuppressWarnings("unchecked")
                              			T item = (T) data[0];
                              
                              			// Initialize and clear:
                              			initializeAllLazyProperties(session);
                              
                              			session.clear();
                              
                              			return item;
                              		}
                              
                              		return null;
                              	}
                              
                              	/**
                              	 * This method retrieves all proxies and all collections from Hibernate persistence context and forces the
                              	 * initialization for them.
                              	 * 
                              	 * @see org.hibernate.Hibernate#initialize(Object)
                              	 * @see org.hibernate.engine.StatefulPersistenceContext#clear()
                              	 */
                              	private static void initializeAllLazyProperties(Session session) {
                              		final StatefulPersistenceContext context = (StatefulPersistenceContext) ((SessionImplementor) session)
                              					.getPersistenceContext();
                              
                              		// During collection loading other collections are registered in the session, so we need:
                              		// * Use array to create a current snapshot of persistence collections
                              		// * Redo the initialization again until there no uninitialized collections.
                              
                              		boolean continueInitialization;
                              
                              		do {
                              			final PersistentCollection[] persistentCollections = (PersistentCollection[]) IteratorUtils.toList(
                              						IdentityMap.keyIterator(context.getCollectionEntries())).toArray(new PersistentCollection[0]);
                              
                              			final HibernateProxy[] proxies = (HibernateProxy[]) context.getProxiesByKey().values()
                              						.toArray(new HibernateProxy[0]);
                              
                              			for (HibernateProxy proxy : proxies) {
                              				proxy.getHibernateLazyInitializer().initialize();
                              			}
                              
                              			continueInitialization = false;
                              
                              			for (PersistentCollection persistentCollection : persistentCollections) {
                              				if (!persistentCollection.wasInitialized()) {
                              					persistentCollection.forceInitialization();
                              					continueInitialization = true;
                              				}
                              			}
                              		}
                              		while (continueInitialization);
                              	}
                              }
                              It would be nice of Batch framework can incorporate any of the solutions.

                              Any feedback is welcomed.
                              Last edited by dma_k; Aug 29th, 2011, 06:28 AM.

                              Comment

                              Working...
                              X