Announcement Announcement Module
Collapse
No announcement yet.
how to append to file in subsequent step? Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • how to append to file in subsequent step?

    I need to create a file that has the general form

    | record | record | record | summary-record |

    The way I have it set up is as a job with two steps, the first step writes the records, and the second step writes the summary record. However, I can't figure out how to have the second step write to the same file as the first step to append to it. I'm using the FlatFileItemWriter, and the only property exposed that is related is whether or not to delete the file if it exists.

    Is there any way to append to a file?

    The only alternative I can think of is to write to a separate file, and add a third step that takes the file created in step 2 and append it to the file from step 1. That wouldn't be ideal but could work I guess.

    Any help is appreciated. Thanks.

  • #2
    Originally posted by raysuliteanu View Post
    The only alternative I can think of is to write to a separate file, and add a third step that takes the file created in step 2 and append it to the file from step 1.
    That's what most people do in this scenario. Or you might be able to write the footer in step 1 using a StepExecutionListener.

    If you can think of a way to make it easier, feel free to suggest something.

    Comment


    • #3
      I've seen a lot of people do this with StepExecutionListener#afterStep(), you just need to use the same FlatFileItemWriter as you did to write the normal records, with the last write being the footer.

      Comment


      • #4
        Thanks. Using an afterStep() implementation works well, better than a separate step.

        Comment


        • #5
          flushing?

          Originally posted by lucasward View Post
          I've seen a lot of people do this with StepExecutionListener#afterStep(), you just need to use the same FlatFileItemWriter as you did to write the normal records, with the last write being the footer.
          It also means you have to call flush on the FlatFileItemWriter in the StepExecutionListener#afterStep(), right? As the resource is still opened, but has already been flushed for the last time as far as the step implementation (ItemOritentedStep) is concerned?

          Comment


          • #6
            I would hope that you could get away with not flushing. The underlying FileOutputStream will flush when it closes, so there's no danger of losing any data there. Are you just curious, or did you find a problem?

            Comment


            • #7
              Originally posted by Dave Syer View Post
              Are you just curious, or did you find a problem?
              Just curious. I was also wondering how to go about having a "checksum" with total numer of rows or something like that. The only way I can think of (that would also support restarts) is to have a DelegateFlatFileReader count rows and set a property in the execution context, which subsequently can be used by the listener.... I haven't actually tried, so it is just speculation at the moment. I'll give it a go later.

              Comment


              • #8
                Restarts are driven through the ItemStream interface, so you just need to implement that to have your cumulative checksum stored by the framework. Remember to also delegate calls to other ItemStreams that you wrap, or else register them with the step.

                Comment

                Working...
                X