Announcement Announcement Module
Collapse
No announcement yet.
hibernate pool configuration Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • hibernate pool configuration

    I apologise in advance for the lengthy post. Below is an edited extract of my Spring configuration file showing one Hibernate object and associated DAO manager. I am using Hibernate 3 with Springs declarative transactions. This seems to work fine but I am experiencing OutOfMemoryError exceptions when displaying large amounts of information retrieved from the database (I am using Tapestry). I am wondering if I should pool the objects fetched from the database or, am I correct in thinking that Hibernate caches these objects anyway and pooling them is a waste of resources. Is there a way to force object eviction straight after a retrieval from the database (only when requested). My question really is how can I reduce the resources my application is using. Is my configuration reasonable if not how should it be configured. The area I am most concerned about is the pooling.

    Thanks in advance,

    Paul


    <!-- datasource (database connection) -->
    <!-- poolable connection -->
    <bean id="cpmsDS" class="org.springframework.jdbc.datasource.DriverM anagerDataSource">
    <property name="driverClassName"><value>com.mysql.jdbc.Drive r</value></property>
    <property name="url"><value>jdbc:mysql://localhost:3306/myDatabase</value></property>
    <property name="username"><value>dud</value></property>
    <property name="password"><value>dud</value></property>
    </bean>

    <bean id="pool" class="org.apache.commons.pool.impl.GenericObjectP ool">
    <property name="minEvictableIdleTimeMillis"><value>300000</value></property>
    <property name="timeBetweenEvictionRunsMillis"><value>60000</value></property>
    </bean>

    <bean id="dsConnectionFactory" class="org.apache.commons.dbcp.DataSourceConnectio nFactory">
    <constructor-arg><ref bean="cpmsDS"/></constructor-arg>
    </bean>

    <bean id="poolableConnectionFactory" class="org.apache.commons.dbcp.PoolableConnectionF actory">
    <constructor-arg index="0"><ref bean="dsConnectionFactory"/></constructor-arg>
    <constructor-arg index="1"><ref bean="pool"/></constructor-arg>
    <constructor-arg index="2"><null/></constructor-arg>
    <constructor-arg index="3"><null/></constructor-arg>
    <constructor-arg index="4"><value>false</value></constructor-arg>
    <constructor-arg index="5"><value>true</value></constructor-arg>
    </bean>

    <bean id="pooledDS" class="org.apache.commons.dbcp.PoolingDataSource" depends-on="poolableConnectionFactory">
    <constructor-arg><ref bean="pool"/></constructor-arg>
    </bean>


    <!-- Hibernate SessionFactory -->
    <bean id="sessionFactory" class="org.springframework.orm.hibernate3.LocalSes sionFactoryBean">
    <property name="dataSource"><ref local="pooledDS"/></property>

    <!-- References all OR mapping files. -->
    <property name="mappingResources">
    <list>
    <value>uk/co/cymbol/tair/db/mapping/job.hbm.xml</value>
    </list>
    </property>

    <!-- Set the type of database; changing this one property will port this to Oracle, MS SQL etc. -->
    <property name="hibernateProperties">
    <props>
    <prop key="hibernate.dialect">org.hibernate.dialect.MySQ LDialect</prop>
    </props>
    </property>
    </bean>

    <!-- Transaction manager for a single Hibernate SessionFactory -->
    <bean id="transactionManager" class="org.springframework.orm.hibernate3.Hibernat eTransactionManager">
    <property name="sessionFactory"><ref local="sessionFactory"/></property>
    </bean>

    <!-- jobMangerDao data access object: Hibernate implementation -->
    <bean id="jobManagerTarget" class="uk.co.cymbol.tair.db.JobManagerDaoImpl">
    <property name="sessionFactory"><ref local="sessionFactory"/></property>
    </bean>


    <!-- parent template for the transaction proxy beans -->
    <bean id="txProxyTemplate" abstract="true" class="org.springframework.transaction.interceptor .TransactionProxyFactoryBean">
    <property name="transactionManager"><ref local="transactionManager"/></property>
    <property name="transactionAttributes">
    <props>
    <prop key="get*">PROPAGATION_REQUIRED,readOnly</prop>
    <prop key="add*">PROPAGATION_REQUIRED</prop>
    <prop key="delete*">PROPAGATION_REQUIRED</prop>
    <prop key="update*">PROPAGATION_REQUIRED</prop>
    </props>
    </property>
    </bean>

    <!-- Transactional proxy for jobManagerDao data access object. -->
    <bean id="jobManagerDao" parent="txProxyTemplate">
    <property name="target"><ref local="jobManagerTarget"/></property>
    </bean>

  • #2
    So you're retrieving large number of mapped objects with Hibernate queries?
    I am wondering if I should pool the objects fetched from the database or, am I correct in thinking that Hibernate caches these objects anyway and pooling them is a waste of resources.
    Yes, Hibernate caches and manages them. Pooling won't do any good.

    Having a large number of objects in the transactional cache can cause problems in Hibernate, in my experience. Techniques I've used include:

    - Projection queries, so you don't retrieve mapped objects and don't increase the size of the session
    - Read-only transactions to avoid the need for expensive dirty checking
    - JDBC-based approach to that use case

    I don't think your Spring config (except for connection pool) is going to make much difference, as all this is dictated by Hibernate usage.

    Rgds
    Rod

    Comment


    • #3
      Thanks Rod for the reponse,

      I am already using projection queries and a read-only transaction. I have also had a go at a JDBC approach.

      Further investigation, looking at my part of the startup trace which is:

      Code:
      09&#58;48&#58;48.846 INFO   &#91;main&#93; org.hibernate.transaction.TransactionFactoryFactory.buildTransactionFactory&#40;TransactionFactoryFactory.java&#58;31&#41; >24> Using default transaction strategy &#40;direct JDBC transactions&#41;
      09&#58;48&#58;48.853 INFO   &#91;main&#93; org.hibernate.transaction.TransactionManagerLookupFactory.getTransactionManagerLookup&#40;TransactionManagerLookupFactory.java&#58;33&#41; >24> No TransactionManagerLookup configured &#40;in JTA environment, use of read-write or transactional second-level cache is not recommended&#41;
      09&#58;48&#58;48.853 INFO   &#91;main&#93; org.hibernate.cfg.SettingsFactory.buildSettings&#40;SettingsFactory.java&#58;126&#41; >22> Automatic flush during beforeCompletion&#40;&#41;&#58; disabled
      09&#58;48&#58;48.854 INFO   &#91;main&#93; org.hibernate.cfg.SettingsFactory.buildSettings&#40;SettingsFactory.java&#58;130&#41; >22> Automatic session close at end of transaction&#58; disabled
      09&#58;48&#58;48.855 INFO   &#91;main&#93; org.hibernate.cfg.SettingsFactory.buildSettings&#40;SettingsFactory.java&#58;137&#41; >22> JDBC batch size&#58; 15
      09&#58;48&#58;48.855 INFO   &#91;main&#93; org.hibernate.cfg.SettingsFactory.buildSettings&#40;SettingsFactory.java&#58;140&#41; >22> JDBC batch updates for versioned data&#58; disabled
      09&#58;48&#58;48.857 INFO   &#91;main&#93; org.hibernate.cfg.SettingsFactory.buildSettings&#40;SettingsFactory.java&#58;145&#41; >22> Scrollable result sets&#58; enabled
      09&#58;48&#58;48.858 INFO   &#91;main&#93; org.hibernate.cfg.SettingsFactory.buildSettings&#40;SettingsFactory.java&#58;153&#41; >22> JDBC3 getGeneratedKeys&#40;&#41;&#58; enabled
      09&#58;48&#58;48.859 INFO   &#91;main&#93; org.hibernate.cfg.SettingsFactory.buildSettings&#40;SettingsFactory.java&#58;161&#41; >22> Connection release mode&#58; null
      09&#58;48&#58;48.861 INFO   &#91;main&#93; org.hibernate.cfg.SettingsFactory.buildSettings&#40;SettingsFactory.java&#58;185&#41; >22> Maximum outer join fetch depth&#58; 2
      09&#58;48&#58;48.862 INFO   &#91;main&#93; org.hibernate.cfg.SettingsFactory.buildSettings&#40;SettingsFactory.java&#58;188&#41; >22> Default batch fetch size&#58; 1
      09&#58;48&#58;48.863 INFO   &#91;main&#93; org.hibernate.cfg.SettingsFactory.buildSettings&#40;SettingsFactory.java&#58;192&#41; >22> Generate SQL with comments&#58; disabled
      09&#58;48&#58;48.863 INFO   &#91;main&#93; org.hibernate.cfg.SettingsFactory.buildSettings&#40;SettingsFactory.java&#58;196&#41; >22> Order SQL updates by primary key&#58; disabled
      09&#58;48&#58;48.864 INFO   &#91;main&#93; org.hibernate.cfg.SettingsFactory.createQueryTranslatorFactory&#40;SettingsFactory.java&#58;321&#41; >23> Query translator&#58; org.hibernate.hql.ast.ASTQueryTranslatorFactory
      09&#58;48&#58;48.881 INFO   &#91;main&#93; org.hibernate.hql.ast.ASTQueryTranslatorFactory.<init>&#40;ASTQueryTranslatorFactory.java&#58;21&#41; >30> Using ASTQueryTranslatorFactory
      09&#58;48&#58;48.882 INFO   &#91;main&#93; org.hibernate.cfg.SettingsFactory.buildSettings&#40;SettingsFactory.java&#58;204&#41; >22> Query language substitutions&#58; &#123;&#125;
      09&#58;48&#58;48.883 INFO   &#91;main&#93; org.hibernate.cfg.SettingsFactory.buildSettings&#40;SettingsFactory.java&#58;210&#41; >22> Second-level cache&#58; enabled
      09&#58;48&#58;48.884 INFO   &#91;main&#93; org.hibernate.cfg.SettingsFactory.buildSettings&#40;SettingsFactory.java&#58;214&#41; >22> Query cache&#58; disabled
      09&#58;48&#58;48.884 INFO   &#91;main&#93; org.hibernate.cfg.SettingsFactory.createCacheProvider&#40;SettingsFactory.java&#58;308&#41; >23> Cache provider&#58; org.hibernate.cache.EhCacheProvider
      09&#58;48&#58;48.890 INFO   &#91;main&#93; org.hibernate.cfg.SettingsFactory.buildSettings&#40;SettingsFactory.java&#58;229&#41; >22> Optimize cache for minimal puts&#58; disabled
      09&#58;48&#58;48.890 INFO   &#91;main&#93; org.hibernate.cfg.SettingsFactory.buildSettings&#40;SettingsFactory.java&#58;238&#41; >22> Structured second-level cache entries&#58; enabled
      09&#58;48&#58;48.902 INFO   &#91;main&#93; org.hibernate.cfg.SettingsFactory.buildSettings&#40;SettingsFactory.java&#58;262&#41; >22> Statistics&#58; disabled
      09&#58;48&#58;48.903 INFO   &#91;main&#93; org.hibernate.cfg.SettingsFactory.buildSettings&#40;SettingsFactory.java&#58;266&#41; >22> Deleted entity synthetic identifier rollback&#58; disabled
      09&#58;48&#58;48.907 INFO   &#91;main&#93; org.hibernate.cfg.SettingsFactory.buildSettings&#40;SettingsFactory.java&#58;280&#41; >22> Default entity-mode&#58; pojo
      09&#58;48&#58;49.141 INFO   &#91;main&#93; org.hibernate.impl.SessionFactoryImpl.<init>&#40;SessionFactoryImpl.java&#58;151&#41; >21> building session factory
      09&#58;48&#58;51.250 INFO   &#91;main&#93; org.hibernate.impl.SessionFactoryObjectFactory.addInstance&#40;SessionFactoryObjectFactory.java&#58;82&#41; >22> Not binding factory to JNDI, no JNDI name configured
      09&#58;48&#58;51.250 INFO   &#91;main&#93; org.hibernate.impl.SessionFactoryImpl.checkNamedQueries&#40;SessionFactoryImpl.java&#58;377&#41; >22> Checking 66 named queries
      09&#58;48&#58;51.845 INFO   &#91;main&#93; org.springframework.orm.hibernate3.HibernateTransactionManager.afterPropertiesSet&#40;HibernateTransactionManager.java&#58;345&#41; >18> Using DataSource &#91;org.apache.commons.dbcp.PoolingDataSource@1e418&#93; of Hibernate SessionFactory for HibernateTransactionManager
      09&#58;48&#58;51.913 INFO   &#91;main&#93; org.springframework.cache.ehcache.EhCacheManagerFactoryBean.afterPropertiesSet&#40;EhCacheManagerFactoryBean.java&#58;69&#41; >18> Initializing EHCache CacheManager
      09&#58;48&#58;51.921 INFO   &#91;main&#93; org.springframework.cache.ehcache.EhCacheFactoryBean.afterPropertiesSet&#40;EhCacheFactoryBean.java&#58;180&#41; >18> Using existing EHCache cache region 'userCache'
      09&#58;48&#58;52.156 INFO   &#91;main&#93; org.springframework.aop.framework.DefaultAopProxyFactory.<clinit>&#40;DefaultAopProxyFactory.java&#58;59&#41; >34> CGLIB2 available&#58; proxyTargetClass feature enabled
      09&#58;48&#58;52.428 INFO   &#91;main&#93; org.springframework.web.context.ContextLoader.initWebApplicationContext&#40;ContextLoader.java&#58;161&#41; >09> Using context class &#91;org.springframework.web.context.support.XmlWebApplicationContext&#93; for root WebApplicationContext
      09&#58;48&#58;52.428 INFO   &#91;main&#93; org.springframework.web.context.ContextLoader.initWebApplicationContext&#40;ContextLoader.java&#58;171&#41; >09> Root WebApplicationContext&#58; initialization completed in 7709 ms

      There are a number of things that concern me.

      1) The EHCache being used is 'userCache' which I set up for Acegi and is:

      <cache name="userCache"
      maxElementsInMemory="300"
      eternal="false"
      timeToIdleSeconds="500"
      timeToLiveSeconds="500"
      overflowToDisk="true"
      />

      This could be one of my problems, since the cache is so small and only supposed to hold user objects. How and where do I use a different EHCache (presumeable in <bean id="sessionFactory" class="org.springframework.orm.hibernate3.LocalSes sionFactoryBean">) in the Spring config?

      2) What would happen if I set "Automatic session close at end of transaction" to enabled, and how would I do it?

      Is there anyway of clearing my Hibernate cache manually after such a transaction using getHibernateTemplate() and getting the session to flush and clear the cache or is this a bad idea and will it completely cleanup. Since ultimately as it stands I am ending up with OutOfMemoryErrors this maybe due to the EHCache setting already mentioned.

      Thanks again,

      Paul

      Comment


      • #4
        Consult the EhCache manual for more information but for acegi you can definy a different cache area and for hibernate you can minimize the cache area.
        Instead of making your code aware of the cache (like you are suggesting at the end of the transaction) why don't you specify a different cache strategy and rules so the cache is invalidated a lot faster or is smaller so the overhead in memory is smaller?

        Comment


        • #5
          Thanks Costin

          I have tried setting a second ehcache up as follows:

          my ehcache.xml file is:

          Code:
          <ehcache>
              <!-- user cache for Acegi Security System -->
              <cache name="userCache"
                  maxElementsInMemory="300"
                  eternal="false"
                  timeToIdleSeconds="500"
                  timeToLiveSeconds="500"
                  overflowToDisk="true"
                  />
          
              <cache name="queryCache"
                  maxElementsInMemory="3"
                  eternal="false"
                  timeToIdleSeconds="5"
                  timeToLiveSeconds="5"
                  overflowToDisk="true"
                  />
          </ehcache>
          my acegi user cache is setup in my spring config file like this:

          Code:
            <!-- caches user objects to enhance user authentication performance &#40;not necessary&#41;
                 if not specified will default to NullUserCache
            -->
            <bean id="cacheManager" class="org.springframework.cache.ehcache.EhCacheManagerFactoryBean">
              <property name="configLocation"><value>classpath&#58;ehcache.xml</value></property>
            </bean>
          
            <bean id="userCacheBackend" class="org.springframework.cache.ehcache.EhCacheFactoryBean">
              <property name="cacheManager"><ref local="cacheManager"/></property>
              <property name="cacheName"><value>userCache</value></property>
            </bean>
            
            <bean id="userCache" class="net.sf.acegisecurity.providers.dao.cache.EhCacheBasedUserCache">
              <property name="cache"><ref local="userCacheBackend"/></property>
            </bean>
          can I not set up a query cache like this:

          Code:
            <bean id="queryCache" class="org.springframework.cache.ehcache.EhCacheFactoryBean">
              <property name="cacheManager"><ref local="cacheManager"/></property>
              <property name="cacheName"><value>queryCache</value></property>
            </bean>
          but I do not know how to reference it in:

          Code:
            <!-- Hibernate SessionFactory -->
            <bean id="sessionFactory" class="org.springframework.orm.hibernate3.LocalSessionFactoryBean">
              <property name="dataSource"><ref local="pooledDS"/></property>
          
              <!-- References all OR mapping files. -->
              <property name="mappingResources">
                <list>
                  <value>uk/co/cymbol/tair/db/mapping/job.hbm.xml</value>
                </list>
              </property>
          
              <!-- Set the type of database; changing this one property will port this to Oracle, MS SQL etc. -->
              <property name="hibernateProperties">
                <props>
                  <prop key="hibernate.dialect">org.hibernate.dialect.MySQLDialect</prop>
                </props>
              </property>
            </bean>
          hibernate always references my userCache.

          Am I going in the right direction? I have not been able to find any references on setting the hibernatProperties to create a secondary ehcache.

          Thanks

          Paul

          Comment


          • #6
            hibernate always references my userCache.
            What do you mean? A problem w/ EhCache is that it support only one Cache Manager - it's a singleton basically. The upcoming version 1.2 (in beta now) I think will solve this issue allowing multiple caching managers.

            Comment


            • #7
              I was hoping to use the same cache manager:

              Code:
                <bean id="cacheManager" class="org.springframework.cache.ehcache.EhCacheManagerFactoryBean">
                  <property name="configLocation"><value>classpath&#58;ehcache.xml</value></property>
                </bean>
              but use a separate cache bean factory:


              Code:
                <bean id="queryCache" class="org.springframework.cache.ehcache.EhCacheFactoryBean">
                  <property name="cacheManager"><ref local="cacheManager"/></property>
                  <property name="cacheName"><value>queryCache</value></property>
                </bean>
              which references the previously created cache manager that is used by acegi this one is the "userCache" that has been configured.

              This is the one that is alway used by hibernate as seen in the startup trace on a previous post.

              Paul

              Comment


              • #8
                Hibernate related properties in the
                Code:
                <bean id="sessionFactory" class="org.springframework.orm.hibernate3.LocalSessionFactoryBean">
                section can be found in the hibernate documentation chapter 4 on configuration.

                Comment

                Working...
                X