Announcement Announcement Module
Collapse
No announcement yet.
Outbound Channel Adapter - Way to generate value for directory path dynamically Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • Outbound Channel Adapter - Way to generate value for directory path dynamically

    Hi all

    I am using Spring Integration to process delimited flat files. Upon completion of processing, would like to move the file to an archive folder (a static value picked up from property file) and delete the original file. Used the following configuration for outbound channel adapter to achive this.
    Code:
    <file:outbound-channel-adapter channel="outputChannel"
    		auto-create-directory="true"
    		delete-source-files="true" 
    		directory="file:${archiveDir.FLATDATA}" />
    Now, would like to enhance the archival process. The archive directory name need to be generated dynamically; fetch the base directory path from the property file as before and append a directory structure based on current year, month, date and day as follows : yyyy/monthNumber-monthName/dayNumeric-dayOfWeek.
    Say if the base path is d:/flatdata/home, the expected archival path would be d:/flatdata/home/2011/10-October/11-Tuesday.

    How do I create the dynamic date based directory path and append it to the static path picked from property file and pass this to the outbound-channel-adapter implementation? Appreciate your advice and suggestions.

    Warm regards
    Chandra

  • #2
    You can use 'filename-generator' which points to a reference of the bean that implements org.springframework.integration.file.FileNameGener ator or 'filename-generator-expression' attribute that allows you to specify a simple SpEL expression which generates the file name.

    Comment


    • #3
      Hello.
      1. yyyy/monthNumber-monthName/dayNumeric-dayOfWeek it is from SimpleDateFormat
      2. How do you read properties file inside Spring?
      3. You can define some PropertiesFactoryBean and add to him your own property based on new Date() and using solution from point 1.

      Artem Bilan

      Comment


      • #4
        oleg / cleric

        Thank you very much for your suggestions.

        I created a new implementation extending DefaultFileNameGenerator. Used SimpleDateFormat to generate a string of the form "2011/10-October/11-Tuesday" and prefixed it to the file name and returned this value. ("2011/10-October/11-Tuesday/DataFile.txt").

        I am using org.springframework.beans.factory.config.PropertyP laceholderConfigurer to read data from property file. My property file contains the following property: archiveDir.FLATDATA=d:/flatdata/home

        Code:
        	<file:outbound-channel-adapter channel="outputChannelFATP"
        		auto-create-directory="true"
        		delete-source-files="true" 
        		directory="file:${datedDirectory}"
                                  filename-generator="filenameGenerator"
        	/>	
        
                    <bean id="filenameGenerator" class="com.flatdata.DatedDirectoryGenerator">
        
        public class DatedDirectoryFactory extends  DefaultFileNameGenerator{
         public String generateFileName(Message<?> message) {
                File inputFile = (File)message.getPayload();
                String originalFileName = super.generateFileName(message);
                return "file:" + file.separatorChar + new SimpleDateFormat("yyyy/MM-MMMMM/dd-EEEE").format(new Date()) + File.separatorChar + originalFileName;
            }
        }
        When I ran the code, the container threw an FileNotFoundException : d:/flatdata/home/2011/10-October/11-Tuesday/DataFile.txt.writing (The system cannot find the path specified).

        Questions:
        1. The auto-create-directory property would create the folders specified against the directory property only. Please confirm.
        2. When I create the dated directory path using SimpleDateFormat and prefix the string with the file name and pass it to the outbound channel adapter, I expected the container to create the directory structure d:/flatdata/home/2011/10-October/11-Tuesday and archive the file. So, is there a feature in Spring modules that help us to create the directory structure dynamically or do we need to create the structure manually using java.io within the FileNameGenerator implementation and create the desired directory structure?
        Please note that the files may be polled on a periodic basis and hence the archival folder path would change every day.

        Regards
        Chandra

        Comment


        • #5
          oleg / cleric

          Thank you very much for your suggestions.

          I created a new implementation extending DefaultFileNameGenerator. Used SimpleDateFormat to generate a string of the form "2011/10-October/11-Tuesday" and prefixed it to the file name and returned this value. ("2011/10-October/11-Tuesday/DataFile.txt").

          I am using org.springframework.beans.factory.config.PropertyP laceholderConfigurer to read data from property file. My property file contains the following property: archiveDir.FLATDATA=d:/flatdata/home

          Code:
          	<file:outbound-channel-adapter channel="outputChannelFATP"
          		auto-create-directory="true"
          		delete-source-files="true" 
          		directory="file:${archiveDir.FLATDATA}"
                                    filename-generator="filenameGenerator"
          	/>	
          
                      <bean id="filenameGenerator" class="com.flatdata.DatedDirectoryGenerator">
          
          public class DatedDirectoryGenerator extends  DefaultFileNameGenerator{
           public String generateFileName(Message<?> message) {
                  File inputFile = (File)message.getPayload();
                  String originalFileName = super.generateFileName(message);
                  return "file:" + file.separatorChar + new SimpleDateFormat("yyyy/MM-MMMMM/dd-EEEE").format(new Date()) + File.separatorChar + originalFileName;
              }
          }
          When I ran the code, the container threw an FileNotFoundException : d:/flatdata/home/2011/10-October/11-Tuesday/DataFile.txt.writing (The system cannot find the path specified).

          Questions:
          1. The auto-create-directory property would create the folders specified against the directory property only. Please confirm.
          2. When I create the dated directory path using SimpleDateFormat and prefix the string with the file name and pass it to the outbound channel adapter, I expected the container to create the directory structure d:/flatdata/home/2011/10-October/11-Tuesday and archive the file. So, is there a feature in Spring modules that help us to create the directory structure dynamically or do we need to create the structure manually using java.io within the FileNameGenerator implementation and create the desired directory structure?
          Please note that the files may be polled on a periodic basis and hence the archival folder path would change every day.

          Regards
          Chandra

          Comment


          • #6
            Dynamic path issue with DefaultFileNameGenerator

            Hi All,

            Sucks, i got same issue... when i generate a file name with DefaultFileNameGenerator which contains "/", spring treats it as new directory and thorws FileNotFound exception

            Questions:
            1. The auto-create-directory property would create the folders specified against the directory property only. Please confirm.
            2. Is there any way to create a directory with DefaultFileNameGenerator?
            appreciate your help
            Thanks in advance

            Originally posted by Chandra_Sivaraman View Post
            oleg / cleric

            Thank you very much for your suggestions.

            I created a new implementation extending DefaultFileNameGenerator. Used SimpleDateFormat to generate a string of the form "2011/10-October/11-Tuesday" and prefixed it to the file name and returned this value. ("2011/10-October/11-Tuesday/DataFile.txt").

            I am using org.springframework.beans.factory.config.PropertyP laceholderConfigurer to read data from property file. My property file contains the following property: archiveDir.FLATDATA=d:/flatdata/home

            Code:
            	<file:outbound-channel-adapter channel="outputChannelFATP"
            		auto-create-directory="true"
            		delete-source-files="true" 
            		directory="file:${archiveDir.FLATDATA}"
                                      filename-generator="filenameGenerator"
            	/>	
            
                        <bean id="filenameGenerator" class="com.flatdata.DatedDirectoryGenerator">
            
            public class DatedDirectoryGenerator extends  DefaultFileNameGenerator{
             public String generateFileName(Message<?> message) {
                    File inputFile = (File)message.getPayload();
                    String originalFileName = super.generateFileName(message);
                    return "file:" + file.separatorChar + new SimpleDateFormat("yyyy/MM-MMMMM/dd-EEEE").format(new Date()) + File.separatorChar + originalFileName;
                }
            }
            When I ran the code, the container threw an FileNotFoundException : d:/flatdata/home/2011/10-October/11-Tuesday/DataFile.txt.writing (The system cannot find the path specified).

            Questions:
            1. The auto-create-directory property would create the folders specified against the directory property only. Please confirm.
            2. When I create the dated directory path using SimpleDateFormat and prefix the string with the file name and pass it to the outbound channel adapter, I expected the container to create the directory structure d:/flatdata/home/2011/10-October/11-Tuesday and archive the file. So, is there a feature in Spring modules that help us to create the directory structure dynamically or do we need to create the structure manually using java.io within the FileNameGenerator implementation and create the desired directory structure?
            Please note that the files may be polled on a periodic basis and hence the archival folder path would change every day.

            Regards
            Chandra

            Comment


            • #7
              Hi!

              Try to use directory-expression instead.

              -Artem

              Comment


              • #8
                Just to explain a little bit further - the adapter won't dynamically create directories in a file name created by the generator; it is expected that is a simple filename, not a path.

                However, when you use a directory-expression, it is evaluated on every message and will make directories as needed.

                Code:
                directory-expression="'/foo/' + @someBean.buildDirectoryFor(payload)"
                Code:
                directory-expression="'/foo/' + headers['whereToPutTheFile']"
                etc, etc.

                Comment


                • #9
                  Directory expression is not supported

                  Hi Gary/Cleric,

                  Thanks for your reply.

                  I dont think directory expression is supported in [int-file:outbound-channel-adapter]. Here i would like to generate a csv or xl file and drop in some X location depending on my requirement.

                  Eg: Root:
                  --- A(SUB DIRECTORY)
                  --- B (SUB DIRECTORY)

                  <int-file:outbound-channel-adapter channel="filesBIChannel"
                  auto-create-directory="true" directory=" '${ROOT DIRECTORY}"
                  filename-generator="filenameGenerator" charset="UTF-8" />

                  I see this root directory is initialized onload, can i customize this?


                  Even if i change to
                  <int-sftp:outbound-channel-adapter channel="filesBIChannel"
                  auto-create-directory="true" directory-expression="${ROOT DIRECTORY}"
                  filename-generator="filenameGenerator" charset="UTF-8"/>


                  I got this exception:
                  cvc-complex-type.2.4.c: The matching wildcard is strict, but no declaration can be found for element 'int-sftp:outbound-channel-adapter'.


                  Thanks
                  Sri

                  Originally posted by Gary Russell View Post
                  Just to explain a little bit further - the adapter won't dynamically create directories in a file name created by the generator; it is expected that is a simple filename, not a path.

                  However, when you use a directory-expression, it is evaluated on every message and will make directories as needed.

                  Code:
                  directory-expression="'/foo/' + @someBean.buildDirectoryFor(payload)"
                  Code:
                  directory-expression="'/foo/' + headers['whereToPutTheFile']"
                  etc, etc.

                  Comment


                  • #10
                    <int-sftp:outbound-channel-adapter channel="filesBIChannel"
                    auto-create-directory="true" directory-expression="${ROOT DIRECTORY}"
                    filename-generator="filenameGenerator" charset="UTF-8"/>
                    SFTP has another attribute - remote-directory-expression

                    Comment


                    • #11
                      directory-expression was added to the file:outbound-channel-adapter in 2.2; the current release is 2.2.4.

                      https://jira.springsource.org/browse/INT-805

                      Comment

                      Working...
                      X