Announcement Announcement Module
Collapse

Spring Modules forum decommissioned in favor of Spring Extensions

As the Spring Modules project has been replaced by the Spring Extensions (http://www.springsource.org/extensions) project, this forum has been decommissioned in favour of Spring Extensions one at:
http://forum.springsource.org/forumdisplay.php?f=44

Please see the Spring Extensions home page for a complete list of current projects in Java, .NET and ActionScript. You can also propose one if you want.

Cheers,
Costin Leau
SpringSource - http://www.SpringSource.com- Spring Training, Consulting, and Support - "From the Source"
http://twitter.com/costinl
See more
See less
[validation] : Bean validator performance issues on large volume of data to validate. Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • [validation] : Bean validator performance issues on large volume of data to validate.

    Hi,

    I'm using the xml based bean validation from the springmodules in a data import feature.
    Basically we use jibx to parse a large XML file (60 Meg+) into javabeans then pass them one a time in a BeanValidator (backed by XML) to finally batch insert the valid ones into a database.

    I'm very happy to have used bean validator because it made us win a lot of time.

    But (and there is always a but, isn't) we have some memory consumption problems.
    I used JProfiler to find out where we ate some much memory.
    To my complete surprise (I was flabbergasted because my first impression was "it must be somewhere in our code") the system was creating millions of BeanWrapperImpl, so fast that some time the garbage collector wasn't able to clear them, leading to outof memory errors.
    I found where in our case :
    DefaultApplicabilityCondition.doCheck() andAbstractBeanCondition.doCheck methods. I tested to replace the use of BeanWrapperImpl by commons-beanutils PropertyUtils (static methods instead of one instance for each property) and our system improved in time and memory comsumption.

    I want to know what you guys think about it...thanks

    Paskos

  • #2
    [Validation] Create a jira bug for this ?

    Should I create a Jira tcket for this ?
    It's really painfull.

    Comment

    Working...
    X