Announcement Announcement Module
Collapse
No announcement yet.
Hibernate + Spring, large amount of data Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • Hibernate + Spring, large amount of data

    I am new to Spring. I succesfully run the Petstore example. I found out, that table data can be paginated on jsp page like this

    PagedListHolder myList = new PagedListHolder(this.petStore.getProductListByCate gory(accountForm.getAccount).getFavouriteCategoryI d()));
    myList.setPageSize(numberOfRows);
    ...

    This question occured to me at once - PagedListHolder takes full list of data. But what if the list is very large? Should I always read all the data, or is there some other way to fetch only needed rows? We expect to use hibernate. Thanks.

  • #2
    Your obviously right if you load all the rows then you could experience performance problems. There are lots of articles out there about Hibernate pagination, I have a google and see what is there. I'm pretty sure this has been covered on this forum as well, would be worth searching here.

    e.g.
    http://blog.hibernate.org/cgi-bin/bl.../08/14#fn.html

    Comment


    • #3
      Originally posted by J_N View Post
      This question occured to me at once - PagedListHolder takes full list of data. But what if the list is very large? Should I always read all the data, or is there some other way to fetch only needed rows? We expect to use hibernate. Thanks.
      In this scenario I would write a hibernate query that would only bring back display values, rather than a collection that holds the a full expanded object and all its associations. Have a read about the HQL select new construct in Hibernate. This is negating the "DREDGE" anti-pattern, which involves dredging up a huge net full of data on the off chance you might need one item within it.

      So a search on something like Customer doesn't bring back all Customers on the database, but brings back the something like the Customer's name, date of birth and importantly the surrogate key. identifier or whatever. If the user clicks on an entry the application uses the identifier to go back and load up more detail.

      One has to wonder though why a user needs a list that potentially allows them to page through ALL the data available. Why not allow them to set a limit, or code a default one ?

      Comment


      • #4
        Hi,

        I hope you can provide me some guidelines. My objective is to handle large volume of data. To do this, I like to apply pagination. i.e. rather than loading the whole bunch of rows (from a select query) into memory, I would like to pass some paramers to hibernate Query, and show results page by page. I'm using hibernate with SP, spring and extrememcomponents.

        Here are my codes:
        [in the action class]

        Context context = new HttpServletRequestContext(getRequest());
        LimitFactory limitFactory = new TableLimitFactory(context);
        Limit limit = new TableLimit(limitFactory);
        metadataList = metadataManager.getMetadatas(metadataWo, limit.getPage()); // The obejctive is to view this metadatalist into JSP page


        [in the service layer]

        public List getMetadatas(final MetadataWo metadataWo, int pageNo) {
        final int defaultRows = 50;
        int startRow = (pageNo - 1) * defaultRows ;
        return dao.getMetadatas(metadataWo, startRow, defaultRows);
        }



        [in the dao layer]

        Query q = null;

        q = session.getNamedQuery("getMetadatasSP");

        q.setString(0, metadataWo.getPsId()).
        setString(1,metadataWo.getMetadataId()).
        setString(2,metadataWo.getDocumentName()).
        setString(3,metadataWo.getLocationCode()).
        setString(4,metaDataObjectType).
        setString(5,metadataWo.getUpdatedBy()).
        setString(6,metadataWo.getBuCode()).
        setString(7,metadataWo.getGxpRelevance()).
        setString(8,metadataWo.getDetailsGxpAssessment()).
        setString(9,metadataWo.getBuRelevance()).
        setString(10,metadataWo.getBuCriticalityAssessment ()).
        setString(11, metadataWo.getFailureConsequence()).
        setString(12, metadataWo.getDetailsFailureConsequence()).
        setString(13, metadataWo.getTestingRequirement()).
        setString(14,metadataWo.getDetailsTestingRequireme nt()).
        setString(15,metadataWo.getTestPerformedApproved() ).
        setString(16, metadataWo.getSystemType()).
        setString(17, metadataWo.getRelatedCoreItem()).
        setString(18, metadataWo.getStatusName()).
        setString(19, metadataWo.getCrId()).
        setString(20,metadataWo.getIsSearchForExport()).
        setString(21, metadataWo.getFromDay()).
        setString(22, metadataWo.getFromMonth()).
        setString(23, metadataWo.getFromYear()).
        setString(24,metadataWo.getToDay()).
        setString(25,metadataWo.getToMonth()).
        setString(26,metadataWo.getToYear());

        q.setFirstResult(startRow);
        q.setMaxResults(endRow);
        result = q.list(); // AT THIS STAGE, THE APPLICATION GETS THE ERROR. CAUGHT BY A TRY-CATCH BLOCK



        As the tomcat console, the following error message is generated:

        java.lang.NullPointerException
        at oracle.jdbc.ttc7.TTCAdapter.newTTCType(TTCAdapter. java:300)
        at oracle.jdbc.ttc7.TTCAdapter.createNonPlsqlTTCColum nArray(TTCAdapter.java:270)
        at oracle.jdbc.ttc7.TTCAdapter.createNonPlsqlTTCDataS et(TTCAdapter.java:231)
        at oracle.jdbc.ttc7.TTC7Protocol.doOall7(TTC7Protocol .java:1937)
        at oracle.jdbc.ttc7.TTC7Protocol.parseExecuteDescribe (TTC7Protocol.java:880)
        at oracle.jdbc.driver.OracleStatement.doExecuteQuery( OracleStatement.java:2516)
        at oracle.jdbc.driver.OracleStatement.doExecuteWithTi meout(OracleStatement.java:2850)
        at oracle.jdbc.driver.OraclePreparedStatement.execute Update(OraclePreparedStatement.java:609)
        at oracle.jdbc.driver.OraclePreparedStatement.execute (OraclePreparedStatement.java:685)
        at org.hibernate.dialect.Oracle9Dialect.getResultSet( Oracle9Dialect.java:278)
        at org.hibernate.jdbc.AbstractBatcher.getResultSet(Ab stractBatcher.java:146)
        at org.hibernate.loader.Loader.getResultSet(Loader.ja va:1666)
        at org.hibernate.loader.Loader.doQuery(Loader.java:66 2)
        at org.hibernate.loader.Loader.doQueryAndInitializeNo nLazyCollections(Loader.java:224)
        at org.hibernate.loader.Loader.doList(Loader.java:214 5)
        at org.hibernate.loader.Loader.listIgnoreQueryCache(L oader.java:2029)
        at org.hibernate.loader.Loader.list(Loader.java:2024)
        at org.hibernate.loader.custom.CustomLoader.list(Cust omLoader.java:111)
        at org.hibernate.impl.SessionImpl.listCustomQuery(Ses sionImpl.java:1655)
        at org.hibernate.impl.AbstractSessionImpl.list(Abstra ctSessionImpl.java:142)
        at org.hibernate.impl.SQLQueryImpl.list(SQLQueryImpl. java:164)
        at com.mh.pep.dao.MetadataDaoHibernate$2.doInHibernat e(MetadataDaoHibernate.java:1216)
        at org.springframework.orm.hibernate3.HibernateTempla te.execute(HibernateTemplate.java:362)
        at org.springframework.orm.hibernate3.HibernateTempla te.executeFind(HibernateTemplate.java:332)
        at com.mh.pep.dao.MetadataDaoHibernate.getMetadatas(M etadataDaoHibernate.java:1136)
        at com.mh.pep.service.MetadataManagerImpl.getMetadata s(MetadataManagerImpl.java:1003)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Nativ e Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(Unknow n Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(Un known Source)
        at java.lang.reflect.Method.invoke(Unknown Source)
        at org.springframework.aop.support.AopUtils.invokeJoi npointUsingReflection(AopUtils.java:280)
        at org.springframework.aop.framework.ReflectiveMethod Invocation.invokeJoinpoint(ReflectiveMethodInvocat ion.java:187)
        at org.springframework.aop.framework.ReflectiveMethod Invocation.proceed(ReflectiveMethodInvocation.java :154)
        at org.springframework.transaction.interceptor.Transa ctionInterceptor.invoke(TransactionInterceptor.jav a:107)
        at org.springframework.aop.framework.ReflectiveMethod Invocation.proceed(ReflectiveMethodInvocation.java :176)
        at org.springframework.aop.framework.JdkDynamicAopPro xy.invoke(JdkDynamicAopProxy.java:210)
        at $Proxy5.getMetadatas(Unknown Source)
        at com.mh.pep.webapp.action.MetadataAction.searchList (MetadataAction.java:748)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Nativ e Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(Unknow n Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(Un known Source)
        at java.lang.reflect.Method.invoke(Unknown Source)
        at com.opensymphony.xwork2.DefaultActionInvocation.in vokeAction(DefaultActionInvocation.java:360)
        at com.opensymphony.xwork2.DefaultActionInvocation.in vokeActionOnly(DefaultActionInvocation.java:228)
        at com.opensymphony.xwork2.DefaultActionInvocation.in voke(DefaultActionInvocation.java:202)
        at com.opensymphony.xwork2.interceptor.DefaultWorkflo wInterceptor.doIntercept(DefaultWorkflowIntercepto r.java:177)
        at com.opensymphony.xwork2.interceptor.MethodFilterIn terceptor.intercept(MethodFilterInterceptor.java:8 6)
        at com.opensymphony.xwork2.DefaultActionInvocation.in voke(DefaultActionInvocation.java:200)
        at com.opensymphony.xwork2.validator.ValidationInterc eptor.doIntercept(ValidationInterceptor.java:115)
        at com.opensymphony.xwork2.interceptor.MethodFilterIn terceptor.intercept(MethodFilterInterceptor.java:8 6)
        at com.opensymphony.xwork2.DefaultActionInvocation.in voke(DefaultActionInvocation.java:200)
        at com.opensymphony.xwork2.interceptor.ConversionErro rInterceptor.intercept(ConversionErrorInterceptor. java:123)
        at com.opensymphony.xwork2.DefaultActionInvocation.in voke(DefaultActionInvocation.java:200)
        at com.opensymphony.xwork2.interceptor.ParametersInte rceptor.intercept(ParametersInterceptor.java:147)
        at com.opensymphony.xwork2.DefaultActionInvocation.in voke(DefaultActionInvocation.java:200)
        at com.opensymphony.xwork2.interceptor.StaticParamete rsInterceptor.intercept(StaticParametersIntercepto r.java:105)
        at com.opensymphony.xwork2.DefaultActionInvocation.in voke(DefaultActionInvocation.java:200)
        at org.apache.struts2.interceptor.FileUploadIntercept or.intercept(FileUploadInterceptor.java:204)
        at com.opensymphony.xwork2.DefaultActionInvocation.in voke(DefaultActionInvocation.java:200)
        at com.opensymphony.xwork2.interceptor.ModelDrivenInt erceptor.intercept(ModelDrivenInterceptor.java:74)
        at com.opensymphony.xwork2.DefaultActionInvocation.in voke(DefaultActionInvocation.java:200)
        at com.opensymphony.xwork2.interceptor.ChainingInterc eptor.intercept(ChainingInterceptor.java:115)
        at com.opensymphony.xwork2.DefaultActionInvocation.in voke(DefaultActionInvocation.java:200)
        at com.opensymphony.xwork2.interceptor.I18nIntercepto r.intercept(I18nInterceptor.java:143)
        at com.opensymphony.xwork2.DefaultActionInvocation.in voke(DefaultActionInvocation.java:200)
        at com.opensymphony.xwork2.interceptor.PrepareInterce ptor.intercept(PrepareInterceptor.java:115)
        at com.opensymphony.xwork2.DefaultActionInvocation.in voke(DefaultActionInvocation.java:200)
        at org.apache.struts2.interceptor.ServletConfigInterc eptor.intercept(ServletConfigInterceptor.java:156)
        at com.opensymphony.xwork2.DefaultActionInvocation.in voke(DefaultActionInvocation.java:200)



        .....


        Can you suggest something?

        Thanks,
        ... Chisty

        Comment


        • #5
          Does it get that error on the first attempt? It would be useful to turn on HQL logging and Spring logging as well.

          Comment

          Working...
          X