Announcement Announcement Module
Collapse
No announcement yet.
MultiResourceItemWriter and itemCountLimitPerResource Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • MultiResourceItemWriter and itemCountLimitPerResource

    Hi all,

    I'm using MultiResourceItemWriter to split the output of a chunk oriented tasklet with commit-interval = itemCountLimitPerResource.

    This was producing the expecting result (several output files with a maximum of itemCountLimitPerResource lines per file), until I set the item read number as a multiple of itemCountLimitPerResource with the following scenario:
    - commit-interval = itemCountLimitPerResource = 5,
    - read a 10 lines file,
    - process
    - write using MultiResourceItemWriter and a FlatFileItemWriter as a delegate.

    This produces 3 output files (and not 2 as I expected): 2 with 5 lines each, and an empty 3rd one. I dug into MultiResourceItemWriter source code, and the
    Code:
    setResourceToDelegate()
    method, which create the file on the filesystem via
    Code:
    file.createNewFile()
    is called on MultiResourceItemWriter.open() and in MultiResourceItemWriter.write() when if
    Code:
    (currentResourceItemCount >= itemCountLimitPerResource)
    Hence, just after writing the last item in the 2nd file, the value of currentResourceItemCount is 5, and the test passes: a 3rd file is created on the filesystem, but nothing will be written into it, because the last input item has been read, processed and written.
    I wonder if this is a bug, or if there is a good reason for this I didn't see.

    Also, I wanted to subclass MultiResourceItemWriter and only override the needed methods, but nearly everything is private (protected access of fields and methods should be the way to go for a framework like Spring Batch). I think
    Code:
    file.createNewFile()
    should only be called if the file is not already opened on MultiResourceItemWriter.write(), and then passed to the delegate writer.

    Did anyone encountered this kind of behavior?

    Spring Batch version: 2.1.1 Release.

    Christian.

  • #2
    I think someone else noticed this but I don't remember a JIRA issue, so please feel free to open one (it seems trivial so expect a fix in 2.1.x).

    Comment


    • #3
      I created a JIRA issue https://jira.springframework.org/browse/BATCH-1618 and proposed a patch.

      Comment

      Working...
      X