Announcement Announcement Module
No announcement yet.
Multiple output files with different format from a single input file Page Title Module
Move Remove Collapse
Conversation Detail Module
  • Filter
  • Time
  • Show
Clear All
new posts

  • Multiple output files with different format from a single input file

    Hi there,
    Iím new to Spring Batch and am currently investigating it with a view to incorporating it into our overnight batch architecture. Iím having difficulties creating a job to write to multiple output files from a single input file and would appreciated any suggestions.

    I have a very large input file, and the requirement is to split it into multiple output files. Each output file has a different file format.

    I can implement this in Spring Batch using multiple steps. Each step has a reader, processor and writer. However for each output file I need to reread the input file from the beginning. For Performance reasons I only want to read the input file once as (a: it is very large and (b: the logic to determine the output file can be very complicated.

    So I was thinking is it possible to define just a reader and processor and then call an item writer manually from the processor?
    something like

    Read in each record

    If records Type = 1
    Then write to file1
    If record Type = 2
    Then write to file2
    If record Type = 3
    Then write to file 3
    If record Type = N
    Then write to file N
    -Input file has one record per row.
    -Each input row has the same format.
    -Logic for which output file to choose can be complicated but is derived from the input fields of the row.
    -Each output file has a different format.

    Is this possible? How would you manually call the writer?
    Would this be the correct or acceptable for Spring Batch? Or would a completely different approach be better? Any suggestions would be appreciated.

  • #2
    I have similar requirements, and in addition if record Type is X, I need to produce another record type Y, and call writerX and writerY respectively. How can I achieve this with SpringBatch?



    • #3
      You can use composing file writers, which basically uses the composite pattern. You set up a writer with a delegates property set to the list of item writers. This is explained in detail in the Spring Batch in Action book. In case you are thinking of using Spring batch I will recommend that you buy the book.