Announcement Announcement Module
Collapse
No announcement yet.
Poll Directory for files Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • Poll Directory for files

    Hi

    I'm new to Spring Integration and I've encountered few obstacles, I'm unable to resolve. I've gone thru latest spring ref. I didn't find solution particular to my solution requirement.

    Requirement:

    Poll a specified input directory periodically
    Get list of files satisfying specified format (I've regular expr. to validate)
    Sort the list of files based on file property like last modified timestamp

    Here is the code/config I've followed to achive this.

    My configuration file (config.xml):
    ================================================== ===============================================
    <?xml version="1.0" encoding="UTF-8"?>
    <beans xmlns="http://www.springframework.org/schema/beans"
    ..............................
    ">

    <bean id="propertiesFile"
    class="org.springframework.beans.factory.config.Pr opertyPlaceholderConfigurer">
    <property name="locations">
    <list>
    <value>File.properties</value>
    </list>
    </property>
    </bean>

    <integration:service-activator
    input-channel="inputChannel" output-channel="outChannel" ref="handler"
    method="process" />

    <file:inbound-channel-adapter id="inputChannel"
    directory="file:${inputDir}" filter="filter"
    scanner="myScanner">
    <integrationoller id="poller" fixed-delay="${polling.interval}" />
    </file:inbound-channel-adapter>

    <file:outbound-channel-adapter id="outChannel"
    delete-source-files="true" directory="${archiveDir}" />

    <bean id="dirHandler" class="org.test.sample.poller.Handler">
    <property name="scanner" ref="myScanner" />
    <property name="inputDir" value="${inputDir}" />
    <property name="job" ref="myJob" />
    <property name="jobLauncher" ref="jobLauncher" />
    </bean>

    <bean id="filter"
    class="org.springframework.integration.file.filter s.RegexPatternFileListFilter">
    <constructor-arg value="${fileName.regex}" />
    </bean>

    <bean id="myScanner" class="org.test.sample.poller.myScanner"/>
    <!-- <bean id="recursiveScanner" class="org.springframework.integration.file.Recurs iveLeafOnlyDirectoryScanner" /> -->

    ...................
    ...................
    </bean>
    ================================================== ===============================================

    The Handler Class call is :

    -------------------------------------------------------------------------------------------------
    public class Handler {

    private MyScanner scanner;
    private String inputDir;

    public void processFile() {
    List<File> listFiles = scanner.listFiles(new File(inputDir), String sortType);
    // process my business logic of the list of files

    }

    public MyScanner getMyScanner() {
    return scanner;
    }

    public void setMyScanner(MyScanner ffaFileSortScanner) {
    this.scanner = scanner;
    }

    public String getInputDir() {
    return inputDir;
    }

    public void setInputDir(String inputDir) {
    this.inputDir = inputDir;
    }
    }

    The MyScanner class is :
    public class MyScanner extends RecursiveLeafOnlyDirectoryScanner implements DirectoryScanner {

    MyScanner() {
    super();
    }

    public File[] listEligibleFiles(String inputDir, String sortType) {
    File[] listFiles = super.listEligibleFiles(new File(inputDir));

    return sort(listEligibleFiles(new File(inputDir)), sortType);
    }

    private File[] sort(File[] inputDir, String sortType) {

    if (sortType.equalsIgnoreCase("Asc")) {
    sortFilesAsc(inputDir);
    }
    if (sortType.equalsIgnoreCase("Desc")) {
    sortFilesDesc(inputDir);
    }
    return inputDir;
    }

    public void sortFilesDesc(File[] files) {
    Arrays.sort(files, new Comparator<File>() {
    public int compare(File o1, File o2) {
    if (o1.lastModified() > o2.lastModified()) {
    return -1;
    } else if (o1.lastModified() < o2.lastModified()) {
    return +1;
    } else {
    return 0;
    }
    }
    });
    }

    public void sortFilesAsc(File[] files) {
    Arrays.sort(files, new Comparator<File>() {
    public int compare(File o1, File o2) {
    if (o1.lastModified() < o2.lastModified()) {
    return -1;
    } else if (o1.lastModified() > o2.lastModified()) {
    return +1;
    } else {
    return 0;
    }
    }
    });
    }
    }
    -------------------------------------------------------------------------------------------------

    The application is run while calling new ClassPathXmlApplicationContext(<config.xml>) from a JobRunner java class.

    The Handler class has process method should have the list of filtered files from the directory. The files in the input folder are filtered by RegexPatternFileListFilter at the in-bound channel adapter level. I've used a scanner (like RecursiveLeafOnlyDirectoryScanner) to get the list of files.

    Questions:

    1. The framework class RegexPatternFileListFilter expects constructor argument value. If I hard-code the regular expression then only this work. But, if I put like: <constructor-arg value="${fileName.regex}" /> as above. It doesnot return any files. If you see I've already define PropertyPlaceConfigurer to File.properties and fileName.regex is defined as well. My requirement is to read from a properties file only. Can you please help me how read the value from a properties file instead of hard-code it?

    2. As my requirement is to get list of files of particular format from an input directory. Since, in-bound-channel adapter returns only one file at a time, so, I've to use a scanner to get a list of files. Also, I need sort the files based on file property like lastModified. Due to this, I've to write a custom scanner extending RecursiveLeafOnlyDirectoryScanner with additional functionality to return ordered(Asc/Desc) list of files. But I think this is duplication of responsibilty as scanner and handler both are configured in the inbound adapter, So, in the Handler bean we don't need to inject the scanner and inputDir again.
    a) Is there any way to get the list of files in the Handler class from the in-bound-channel without need to inject inputDir and scanner in the Handler bean again?
    It would be very helpfull if you provide the code snippet to achieve that.

    b) The use of the custom scanner, MyScanner above gives me error:
    --------------------------------------------------------
    INFO : org.springframework.scheduling.concurrent.ThreadPo olTaskScheduler - Shutting down ExecutorService 'taskScheduler'
    ERROR: org.test.sample.poller.JobRunner - Error in runJob method: Error creating bean with name 'org.springframework.integration.config.ServiceAct ivatorFactoryBean#0':
    Cannot resolve reference to bean 'handler' while setting bean property 'targetObject'; nested exception is org.springframework.beans.factory.BeanCreationExce ption:
    Error creating bean with name 'handler' defined in class path resource [config.xml]: Initialization of bean failed;
    nested exception is org.springframework.beans.ConversionNotSupportedEx ception:
    Failed to convert property value of type '$Proxy12 implementing org.springframework.integration.file.DirectoryScan ner,
    org.springframework.aop.SpringProxy,org.springfram ework.aop.framework.Advised' to required type 'org.test.sample.poller.myScanner' for property 'scanner';
    nested exception is java.lang.IllegalStateException: Cannot convert value of type [$Proxy12 implementing org.springframework.integration.file.DirectoryScan ner,
    org.springframework.aop.SpringProxy,org.springfram ework.aop.framework.Advised] to required type [org.test.sample.poller.myScanner] for property 'scanner':
    no matching editors or conversion strategy found java.lang.IllegalStateException:
    Cannot convert value of type [$Proxy12 implementing org.springframework.integration.file.DirectoryScan ner,
    org.springframework.aop.SpringProxy,org.springfram ework.aop.framework.Advised] to required type [org.test.sample.poller.myScanner] for property 'scanner':
    no matching editors or conversion strategy found
    ------------------------------------------------------

    What is mistake I've done in writing the scanner? why the spring integration framework in-bound adapter is not recognizing it?

    Any suggestion or better approach without required to change my requirement is also highly appreciated.

    Appreciate your assistance.
    Pankaj

  • #2
    it might not be finding your properties file. Read more on PropertyPlaceholders here: http://static.springsource.org/sprin...lderconfigurer You can also use a simpler namespace-based configuration:

    Code:
    <context:property-placeholder location="file:/path/to/the/foo.properties"/>
    
    or 
    
    <context:property-placeholder location="classpath:com/foo/bar/foo.properties"/>

    Comment


    • #3
      Thanks, Oleg!

      I've observed that the org.springframework.integration.file.filters.Regex PatternFileListFilter accepts only as constructor-arg and if it accepts value from a properties file only as for example '\\d' not as '\d'.

      However, I'm still unable to convert the MyScanner class to be recognized by the spring fwk. It is throwing org.springframework.beans.ConversionNotSupportedEx ception when injecting my scanner to my Handler class. The details stack trace is:
      ERROR: org.test.sample.poller.JobRunner - Error in runJob method: Error creating bean with name 'org.springframework.integration.config.ServiceAct ivatorFactoryBean#0': Cannot resolve reference to bean 'handler' while setting bean property 'targetObject'; nested exception is org.springframework.beans.factory.BeanCreationExce ption: Error creating bean with name 'handler' defined in class path resource [config.xml]: Initialization of bean failed; nested exception is org.springframework.beans.ConversionNotSupportedEx ception: Failed to convert property value of type '$Proxy11 implementing org.springframework.integration.file.DirectoryScan ner,org.springframework.aop.SpringProxy,org.spring framework.aop.framework.Advised' to required type 'org.test.sample.poller.myScanner' for property 'scanner'; nested exception is java.lang.IllegalStateException: Cannot convert value of type [$Proxy11 implementing org.springframework.integration.file.DirectoryScan ner,org.springframework.aop.SpringProxy,org.spring framework.aop.framework.Advised] to required type [org.test.sample.poller.myScanner] for property 'scanner': no matching editors or conversion strategy found java.lang.IllegalStateException: Cannot convert value of type [$Proxy11 implementing org.springframework.integration.file.DirectoryScan ner,org.springframework.aop.SpringProxy,org.spring framework.aop.framework.Advised] to required type [org.test.sample.poller.myScanner] for property 'scanner': no matching editors or conversion strategy found
      org.springframework.beans.factory.BeanCreationExce ption: Error creating bean with name 'org.springframework.integration.config.ServiceAct ivatorFactoryBean#0': Cannot resolve reference to bean 'dirHandler' while setting bean property 'targetObject'; nested exception is org.springframework.beans.factory.BeanCreationExce ption: Error creating bean with name 'handler' defined in class path resource [config.xml]: Initialization of bean failed; nested exception is org.springframework.beans.ConversionNotSupportedEx ception: Failed to convert property value of type '$Proxy11 implementing org.springframework.integration.file.DirectoryScan ner,org.springframework.aop.SpringProxy,org.spring framework.aop.framework.Advised' to required type 'org.test.sample.poller.myScanner' for property 'scanner'; nested exception is java.lang.IllegalStateException: Cannot convert value of type [$Proxy11 implementing org.springframework.integration.file.DirectoryScan ner,org.springframework.aop.SpringProxy,org.spring framework.aop.framework.Advised] to required type [org.test.sample.poller.myScanner] for property 'scanner': no matching editors or conversion strategy found
      at org.springframework.beans.factory.support.BeanDefi nitionValueResolver.resolveReference(BeanDefinitio nValueResolver.java:328)
      at org.springframework.beans.factory.support.BeanDefi nitionValueResolver.resolveValueIfNecessary(BeanDe finitionValueResolver.java:106)
      at org.springframework.beans.factory.support.Abstract AutowireCapableBeanFactory.applyPropertyValues(Abs tractAutowireCapableBeanFactory.java:1325)
      at org.springframework.beans.factory.support.Abstract AutowireCapableBeanFactory.populateBean(AbstractAu towireCapableBeanFactory.java:1086)
      at org.springframework.beans.factory.support.Abstract AutowireCapableBeanFactory.doCreateBean(AbstractAu towireCapableBeanFactory.java:517)
      ...

      Caused by: org.springframework.beans.factory.BeanCreationExce ption: Error creating bean with name 'handler' defined in class path resource [config.xml]: Initialization of bean failed; nested exception is org.springframework.beans.ConversionNotSupportedEx ception: Failed to convert property value of type '$Proxy11 implementing org.springframework.integration.file.DirectoryScan ner,org.springframework.aop.SpringProxy,org.spring framework.aop.framework.Advised' to required type 'org.test.sample.poller.myScanner' for property 'scanner'; nested exception is java.lang.IllegalStateException: Cannot convert value of type [$Proxy11 implementing org.springframework.integration.file.DirectoryScan ner,org.springframework.aop.SpringProxy,org.spring framework.aop.framework.Advised] to required type [org.test.sample.poller.myScanner] for property 'scanner': no matching editors or conversion strategy found
      at org.springframework.beans.factory.support.Abstract AutowireCapableBeanFactory.doCreateBean(AbstractAu towireCapableBeanFactory.java:527)
      at org.springframework.beans.factory.support.Abstract AutowireCapableBeanFactory.createBean(AbstractAuto wireCapableBeanFactory.java:456)
      at org.springframework.beans.factory.support.Abstract BeanFactory$1.getObject(AbstractBeanFactory.java:2 91)
      at org.springframework.beans.factory.support.DefaultS ingletonBeanRegistry.getSingleton(DefaultSingleton BeanRegistry.java:222)
      at org.springframework.beans.factory.support.Abstract BeanFactory.doGetBean(AbstractBeanFactory.java:288 )
      at org.springframework.beans.factory.support.Abstract BeanFactory.getBean(AbstractBeanFactory.java:190)
      at org.springframework.beans.factory.support.BeanDefi nitionValueResolver.resolveReference(BeanDefinitio nValueResolver.java:322)
      ... 40 more
      Caused by: org.springframework.beans.ConversionNotSupportedEx ception: Failed to convert property value of type '$Proxy11 implementing org.springframework.integration.file.DirectoryScan ner,org.springframework.aop.SpringProxy,org.spring framework.aop.framework.Advised' to required type 'org.test.sample.poller.myScanner' for property 'scanner'; nested exception is java.lang.IllegalStateException: Cannot convert value of type [$Proxy11 implementing org.springframework.integration.file.DirectoryScan ner,org.springframework.aop.SpringProxy,org.spring framework.aop.framework.Advised] to required type [org.test.sample.poller.myScanner] for property 'scanner': no matching editors or conversion strategy found
      at org.springframework.beans.BeanWrapperImpl.convertI fNecessary(BeanWrapperImpl.java:462)
      at org.springframework.beans.BeanWrapperImpl.convertF orProperty(BeanWrapperImpl.java:499)
      at org.springframework.beans.BeanWrapperImpl.convertF orProperty(BeanWrapperImpl.java:493)
      at org.springframework.beans.factory.support.Abstract AutowireCapableBeanFactory.convertForProperty(Abst ractAutowireCapableBeanFactory.java:1371)
      at org.springframework.beans.factory.support.Abstract AutowireCapableBeanFactory.applyPropertyValues(Abs tractAutowireCapableBeanFactory.java:1330)
      at org.springframework.beans.factory.support.Abstract AutowireCapableBeanFactory.populateBean(AbstractAu towireCapableBeanFactory.java:1086)
      at org.springframework.beans.factory.support.Abstract AutowireCapableBeanFactory.doCreateBean(AbstractAu towireCapableBeanFactory.java:517)INFO : org.test.sample.poller.test.JobRunner - End: runJob method for config.xml

      Any clue to make my scanner recognizable to the Spring Integration Framework ?

      I would really appreciate for any help at this.

      Warm Regards,
      Pankaj

      Comment


      • #4
        Do you have any AOP configuration in your config.xml? Can you post your updated configuration?

        Comment

        Working...
        X