Announcement Announcement Module
Collapse
No announcement yet.
Writing data to Multiple file Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • Writing data to Multiple file

    I have one scenario-
    1) Reader read
    a) Company data from database.
    b) Add this data to CompanyDTO.
    c) Add this DTO to ListItemReader.

    This CompanyDTO is then sent to writer one by one to the writer.
    At the writer side corresponding to each CompanyDTO a separate Flat file has to be created, example companyData-01.txt, companyData-02.txt etc... \the number of txt file will depend on the number of CompanyDTO. i.e. corresponding to each DTO a separate file needs to be created.

    I tried using Composite Item Writer in order to print this data to a flat file using FlatFileItemWriter but each and every time one file is getting generated and all DTO data is coming in one file instead of separate file.
    Last edited by harit.dhiman; Dec 1st, 2009, 11:14 AM.

  • #2
    Are you using the built-in CompositeItemWriter class? That one is intended to write the same thing to every file. You need to create a custom composite ItemWriter that delegates the right information to the right file.

    Comment


    • #3
      Originally posted by DHGarrette View Post
      Are you using the built-in CompositeItemWriter class? That one is intended to write the same thing to every file. You need to create a custom composite ItemWriter that delegates the right information to the right file.
      The output file needs to be created dynamically. Its not that i am having 2 FlatFileItemWriter and data should be printed to these file. File should be created dynamically. I am using my own composite itemwriter and using delegator to print data to file but all data is getting printed to one file. New file is not getting created.

      At the writer side corresponding to each CompanyDTO a separate Flat file has to be created, example companyData-01.txt, companyData-02.txt etc... the number of txt file will depend on the number of CompanyDTO. i.e. corresponding to each DTO a separate file needs to be created.

      Comment


      • #4
        Oh, I see now. Unfortunately it's a bit hard to see what's wrong without the code. Can you post your composite ItemWriter?

        Comment


        • #5
          Originally posted by DHGarrette View Post
          Oh, I see now. Unfortunately it's a bit hard to see what's wrong without the code. Can you post your composite ItemWriter?
          I dont have code right now. I can try to explain what we are doing -
          1) DTO has List<attribsA>, attribB. Now in order to print List<AttribsA> we have a FlatFileItemWriter. And in order to print attribB object we have another FlatFileItemWriter. Both of these writer are pointing to same output file.
          Now in write method if i try to open the related writers, write the data and try to close the writers, i am getting error something like 'writer already closed'.
          My main query is:
          How can i create multiple output file with different names such as companyData-01.txt, companyData-02.txt etc....

          Comment


          • #6
            Both of these writer are pointing to same output file.
            This won't work. Each file should have exactly one associated ItemWriter. However, there should be no problem sending both objects to the same ItemWriter.

            Now in write method if i try to open the related writers, write the data and try to close the writers, i am getting error something like 'writer already closed'.
            This seems fine in principle, so it must be an issue with your code. The write() method will be like this:
            Code:
            int fileNumber = 0;
            
            write(List<? extends MyDTO> items) {
                for(MyDTO item : items) {
                    FlatFileItemWriter<Object> w = new FlatFileItemWriter<Object>();
                    w.setResource(baseFilename + (++fileNumber));
                    w..... //set other properties
                    w.open(new ExecutionContext());
                    for(AttribsA a : item.getAttribsA()) {
                        w.write(a);
                    }
                    w.write(item.getAttribB());
                    w.close();
                }
            }

            Comment


            • #7
              Thanks...I will check this out and update you respectively.

              Comment


              • #8
                Originally posted by DHGarrette View Post
                This won't work. Each file should have exactly one associated ItemWriter. However, there should be no problem sending both objects to the same ItemWriter.


                This seems fine in principle, so it must be an issue with your code. The write() method will be like this:
                Code:
                int fileNumber = 0;
                
                write(List<? extends MyDTO> items) {
                    for(MyDTO item : items) {
                        FlatFileItemWriter<Object> w = new FlatFileItemWriter<Object>();
                        w.setResource(baseFilename + (++fileNumber));
                        w..... //set other properties
                        w.open(new ExecutionContext());
                        for(AttribsA a : item.getAttribsA()) {
                            w.write(a);
                        }
                        w.write(item.getAttribB());
                        w.close();
                    }
                }
                I tried above changes but still i am getting the error message. Please find attached
                1) error.txt for error message.
                2) writer.txt - Containing writer snippet.
                3) xml snippet.txt - Step configured in batch specific xml file. recordAAttribsWriter and recordBAttribsWriter are two flat file item writer. In the example we are reading fileName from property file and in writer i am appending it with date (Its just for example, to have distinct file name), but still i am getting the attached error message.
                Last edited by harit.dhiman; Dec 2nd, 2009, 06:41 AM. Reason: Added more details.

                Comment


                • #9
                  recordAAttribsWriter.setResource(resource);
                  recordBAttribsWriter.setResource(resource);
                  As I mentioned before, you cannot have more than one writer for a single file. You either need to have a different file, OR use the same writer for both As and Bs.

                  Comment


                  • #10
                    Originally posted by DHGarrette View Post
                    As I mentioned before, you cannot have more than one writer for a single file. You either need to have a different file, OR use the same writer for both As and Bs.
                    I modified the code. Used one writer...but still i am getting same error...Stream already closed. Is it a bug in Spring framework that we cannot have multiple out put files....Please suggest?

                    Comment


                    • #11
                      There is no bug. This is a common use case. Can you post your revised code?

                      Comment


                      • #12
                        Originally posted by DHGarrette View Post
                        There is no bug. This is a common use case. Can you post your revised code?
                        Please find attached snippet of modified writer. I have configured batch in order to have 1 writer and print only AttribA list. But my requirement is each List of AttribA should get printed in separate output file. But still i am getting same message 'Item strean already closed...........'

                        Comment


                        • #13
                          You need to get rid of the CompositeItemStream stuff. The CompositeItemStream is trying to manage your FFIW for you. However, this is not what you want. Inside your write() method you are manually opening and closing the FFIW. You do not want the framework to try to open/update/close the FFIW.

                          Comment


                          • #14
                            Originally posted by DHGarrette View Post
                            You need to get rid of the CompositeItemStream stuff. The CompositeItemStream is trying to manage your FFIW for you. However, this is not what you want. Inside your write() method you are manually opening and closing the FFIW. You do not want the framework to try to open/update/close the FFIW.
                            My main intention to open and close the FFIW is to make sure each and every time a new file is getting generated. is there any other way to achieve the same.

                            Comment


                            • #15
                              That's exactly what the code sample I provided before will do. It is saying that for each item, create a new FFIW, set a new resource (with a new filename), open it, write to it, and close it. All of which is handled in the write() method.

                              Comment

                              Working...
                              X