Announcement Announcement Module
Collapse
No announcement yet.
Memory leak in spring batch 1.1.4 BatchSqlUpdateItemWriter Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • Memory leak in spring batch 1.1.4 BatchSqlUpdateItemWriter

    Hi guys,

    i use spring batch 1.1.4

    java version "1.5.0_19"
    Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_19-b02)
    Java HotSpot(TM) Server VM (build 1.5.0_19-b02, mixed mode)

    I running my JVM with -Xms10m -Xmx64m

    My batch application reads text file and writes it to database using
    org.springframework.batch.item.file.FlatFileItemRe ader
    and
    org.springframework.batch.item.database.BatchSqlUp dateItemWriter
    in skipLimitStep
    with commit-interval=1000.

    When my input CSV file contains data which causes database constraint violations,
    JVM finally throws OutOfMemory error.

    In Eclipse Memory Analyser i investigated that
    org.springframework.batch.item.database.AbstractTr ansactionalResourceItemWriter
    accumulates sql batch update records in
    private Set failed = new HashSet();
    (see AbstractTransactionalResourceItemWriter.flush() method) every time when butch update fails.
    But neither AbstractTransactionalResourceItemWriter nor BatchSqlUpdateItemWriter never flushes this private Set failed variable.

    There is not any registered issue for this bug in http://opensource.atlassian.com/proj...g/browse/BATCH

    Is this bug will be fixed in spring batch 1.1.x?

    Is this bug fixed in spring batch 2.0.x or 2.1.x?

    Thank you.
    Last edited by slade; Aug 5th, 2010, 01:37 AM.

  • #2
    Neither AbstractTransactionalResourceItemWriter nor BatchSqlUpdateItemWriter are part of the framework in 2.x, so I suggest you upgrade. If you need bug fixes in 1.1.4 I think we still support it commercially for about another year or so. It might not be a bug though, or you can work around it at least, by flushing in a listener (probably).

    Comment


    • #3
      Neither AbstractTransactionalResourceItemWriter nor BatchSqlUpdateItemWriter are part of the framework in 2.x
      Thank you for good news.
      It might not be a bug though, or you can work around it at least, by flushing in a listener (probably).
      Yes, probably "failed" variable is used for rollbacks somehow...
      Unfortunately it's impossible to flush it in listener, AbstractTransactionalResourceItemWriter doesn't have public/protected method for accessing it.

      Comment

      Working...
      X