Announcement Announcement Module
No announcement yet.
Read Multiple files in the same step (flatFileItemReader?) Page Title Module
Move Remove Collapse
Conversation Detail Module
  • Filter
  • Time
  • Show
Clear All
new posts

  • Read Multiple files in the same step (flatFileItemReader?)

    Hi to all.. I'm newbie in spring batch and need some help.. hope could be simple for you guys..

    I'm trying to read multiple resources (files) to create a message to be writen in a database. I cannot process the files in separated steps because they have data dependencies between the registers. Basicly line 1 in file1 plus line 1 in file2 is my register. So in a step processing i should have line n from all files(resources).

    First I was thinking that my solution was creating a MultiResourceItemReader bean:

    <bean id="multiReader" class="org.springframework.batch.item.file.MultiRe sourceItemReader">
    <property name="resources" value="${input.file.list}" />
    <property name="delegate" ref="simpleReader" />
    <property name="saveState" value="true" />

    <bean id="simpleReader" class="org.springframework.batch.item.file.FlatFil eItemReader">
    <property name="resource" value="${input.file.list}" />
    <property name="lineTokenizer" ref="fileRecordTypes" />
    <property name="fieldSetMapper" ref="messageMapper" />

    But in this case my simpleReader process each line separated, sequentialy for all files. What I need is quite different and I did not find any similar situation..

    Could someone help me on that..

    Thanks and advance..

  • #2
    If I understand your usecase correctly you need to implement a composite reader that reads a line from all necessary files (using one FlatFileItemReader per file) and combines those into an item. This is not provided out of box, but shouldn't be hard to do I think.


    • #3
      I was thinking to try something like..

      flatFileItemReader1 ---__
      --- ItemWriter
      flatFileItemReader2 __---

      And in the writer I could treat the msg content from all ItemReader. I'm not sure if thats is possible.. but treats more them one resource in the same ItemReader is not that crazy need, i hope

      Temporary table is in my pipe for the first solution..
      Any ideias?
      thanks a lot



      • #4
        I also got a similar set of requirement to read multiple text/csv files and achieved by using org.springframework.batch.item.file.MultiResourceI temReader.
        The detailed implementation is provided in the below link.