Announcement Announcement Module
Collapse
No announcement yet.
FTP Adapter Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • FTP Adapter

    Hi guys, anyone used FTP adapter? I'm struggling to configure the adapter.

    How does one set the port, username etc?

    I see theres a FTPFileConfig object referenced in the DefaultFTPClientFactory, just not so clear on where what goes where. Just a pointer or two please.


    I found these notes on the FTP adapter ext wiki:
    I'm just not so certain how to configure the DefaultFTPClientFactory injection.

    Code:
    <integration:inbound-channel-adapter
    	channel="file-channel" ref="ftpFileSource" />
    
    <beans:bean id="ftpFileSource"
    	class="org.springframework.integration.ftp.FtpFileSource">
    	<beans:property name="clientPool" ref="ftpClientPool" />
    	<beans:property name="localWorkingDirectory"
    	value="file:${java.io.tmpdir}/FtpConfigIntegrationTests" />
    	<beans:property name="taskScheduler" ref="taskScheduler" />
    	<beans:property name="trigger">
    		<beans:bean
    			class="org.springframework.integration.scheduling.IntervalTrigger">
    			<beans:constructor-arg value="1500" />
    		</beans:bean>
    	</beans:property>
    </beans:bean>
    
    <beans:bean id="ftpClientPool"
    	class="org.springframework.integration.ftp.QueuedFTPClientPool">
    	<beans:constructor-arg>
    		<beans:bean class="org.springframework.integration.ftp.DefaultFTPClientFactory"/>
    	</beans:constructor-arg>
    </beans:bean>

  • #2
    ok managed to get this, but now nothings happening, Spring just hangs...

    I had to include the commons-net jar, I'll add the commons-ftp jar too and see what happens

    Or perhaps my config is wrong?

    Code:
    for now I'm just setting the props on the DefaultFTPClientFactory like so..
    
    
    <channel id="file-channel"/>
    <integration:inbound-channel-adapter
    	channel="file-channel" ref="ftpFileSource" />
    
    <beans:bean id="ftpFileSource"
    	class="org.springframework.integration.ftp.FtpFileSource">
    	<beans:property name="clientPool" ref="ftpClientPool" />
    	<beans:property name="localWorkingDirectory"
    	value="file:${java.io.tmpdir}/FtpConfigIntegrationTests" />
    	<beans:property name="taskScheduler" ref="taskScheduler" />
    	<beans:property name="trigger">
    		<beans:bean
    			class="org.springframework.integration.scheduling.IntervalTrigger">
    			<beans:constructor-arg value="1500" />
    		</beans:bean>
    	</beans:property>
    </beans:bean>
    
    <beans:bean id="ftpClientPool"
    	class="org.springframework.integration.ftp.QueuedFTPClientPool">
    	<beans:constructor-arg>
    		<beans:bean class="org.springframework.integration.ftp.DefaultFTPClientFactory">
    			<beans:property name="host" value="10.182.198.333"/>
    			<beans:property name="port" value="23"/>
    			<beans:property name="username" value="user"/>
    			<beans:property name="password" value="password"/>
    			<!-- beans:property name="remoteWorkingDirectory"/-->
    		</beans:bean>
    	</beans:constructor-arg>
    </beans:bean>
    	
    	<integration:poller id="poller" default="true">
    			<integration:interval-trigger interval="10000"/>
    	</integration:poller>

    Comment


    • #3
      seems it hangs at the taskScheduler

      Code:
      
      DEBUG - AbstractAutowireCapableBeanFactory$1.run(383) | Creating instance of bean 'org.springframework.integration.endpoint.EventDrivenConsumer#0'
      DEBUG - AbstractBeanFactory.doGetBean(214) | Returning cached instance of singleton bean 'errorChannel'
      DEBUG - AbstractBeanFactory.doGetBean(214) | Returning cached instance of singleton bean 'org.springframework.integration.handler.LoggingHandler#0'
      DEBUG - AbstractAutowireCapableBeanFactory.doCreateBean(459) | Eagerly caching bean 'org.springframework.integration.endpoint.EventDrivenConsumer#0' to allow for resolving potential circular references
      DEBUG - AbstractBeanFactory.doGetBean(214) | Returning cached instance of singleton bean 'taskScheduler'
      DEBUG - AbstractAutowireCapableBeanFactory.invokeInitMethods(1367) | Invoking afterPropertiesSet() on bean with name 'org.springframework.integration.endpoint.EventDrivenConsumer#0'
      INFO - AbstractEndpoint.start(87) | started org.springframework.integration.endpoint.EventDrivenConsumer#0
      DEBUG - AbstractAutowireCapableBeanFactory$1.run(411) | Finished creating instance of bean 'org.springframework.integration.endpoint.EventDrivenConsumer#0'
      DEBUG - AbstractBeanFactory.doGetBean(214) | Returning cached instance of singleton bean 'org.springframework.integration.channel.MessagePublishingErrorHandler#0'
      DEBUG - AbstractBeanFactory.doGetBean(214) | Returning cached instance of singleton bean 'taskScheduler'
      INFO - SimpleTaskScheduler.start(137) | started [email protected]5bdb

      Comment


      • #4
        narrowed it down to the fact that I also need to inject a FTPClient config. Not sure how this is done.....hmmm, trawling for docs

        ok referencing the config property now but still hanging in the same place...anyone, pls help, someone must have done this before...?
        Last edited by dudleygb; Sep 18th, 2009, 01:16 PM.

        Comment


        • #5
          now i'm really stuck, its still hanging on the TaskScheduler line as above...this is my config so far, pls help anyone...pls pls

          Code:
          <integration:channel id="file-channel"/>
          <integration:inbound-channel-adapter
          	channel="file-channel" ref="ftpFileSource" />
          
          <bean id="ftpFileSource"
          	class="org.springframework.integration.ftp.FtpFileSource">
          	<property name="clientPool" ref="ftpClientPool" />
          	<property name="localWorkingDirectory"
          	value="file:${java.io.tmpdir}/FtpConfigIntegrationTests" />
          	<property name="taskScheduler" ref="taskScheduler" />
          	<property name="trigger">
          		<bean class="org.springframework.integration.scheduling.IntervalTrigger">
          			<constructor-arg value="1500" />
          		</bean>
          	</property>
          </bean>
          
          <bean id="ftpClientConf"
          	class="org.apache.commons.net.ftp.FTPClientConfig">
          	<constructor-arg value="UNIX" />
          </bean>
          
          <bean id="ftpClientPool"
          	class="org.springframework.integration.ftp.QueuedFTPClientPool">
          	<constructor-arg>
          		<bean class="org.springframework.integration.ftp.DefaultFTPClientFactory">
          			<property name="host" value="localhost"/>
          			<property name="port" value="21"/>
          			<property name="username" value="ftpuser"/>
          			<property name="password" value="passsword"/>
          			<property name="config" ref="ftpClientConf"/>
          			<!-- property name="remoteWorkingDirectory" value=".\"/-->
          		</bean>
          	</constructor-arg>
          </bean>
          	
          	<integration:poller id="poller" default="true">
          			<integration:interval-trigger interval="10000"/>
          	</integration:poller>

          Comment


          • #6
            Code:
               <bean id="ftpClientPool"
                      class="org.springframework.integration.ftp.QueuedFTPClientPool">
                    <constructor-arg>
                        <bean class="org.springframework.integration.ftp.DefaultFTPClientFactory">
                            <property name="host" value="${oad.ftp.host}"/>
                            <property name="username" value="${oad.ftp.username}"/>
                            <property name="password" value="${oad.ftp.password}"/>
                            <property name="port" value="${oad.ftp.port}"/>
                            <property name="remoteWorkingDirectory" value="${oad.ftp.remotedir}"/>
                        </bean>
                    </constructor-arg>
                </bean>
            Works for me.
            Last edited by iwein; Sep 18th, 2009, 02:50 PM. Reason: vBulletin annoyance

            Comment


            • #7
              thanx iwein, must be something else then...
              can u pls post what the ouput looks like when it connects to FTP server and copies a file pls? Its just not working for me, I have no idea what the deal is..

              Comment


              • #8
                I don't have that at hand right now, but maybe you're reading from the wrong remote directory? You can also look at the local directory, the FtpInboundSynchronizer should replicate the contents of the ftp location there, or die horribly screaming about all that went wrong. If the contents is [] it could be a bit silent.

                Comment


                • #9
                  cool thanx for that tip, I placed a file in my local working directory and it is blowing some exceptions as below. Also, as a test I put in a wrong username and password in my DefaultFTPClientFactory and it gives me the same error as below. Almost as if its not actually creating the remote connection.

                  DEBUG - FileReadingMessageSource.scanInputDirectory(201) | Added to queue: [/tmp/FtpConfigIntegrationTests/hello.txt]
                  INFO - FileReadingMessageSource.receive(184) | Created message: [[Payload=/tmp/FtpConfigIntegrationTests/hello.txt][Headers={springintegration_timestamp=1253370628319 , springintegration_id=1437c49f-0995-43cb-b438-e45d24e90a5d}]]
                  DEBUG - AbstractMessageChannel$ChannelInterceptorList.preS end(156) | preSend on channel 'file-channel', message: [Payload=/tmp/FtpConfigIntegrationTests/hello.txt][Headers={springintegration_timestamp=1253370628319 , springintegration_id=1437c49f-0995-43cb-b438-e45d24e90a5d}]
                  DEBUG - AbstractMessageChannel$ChannelInterceptorList.preS end(156) | preSend on channel 'errorChannel', message: [Payload=org.springframework.integration.message.Me ssageDeliveryException: Dispatcher has no subscribers.][Headers={springintegration_timestamp=1253370628320 , springintegration_id=b1218469-e9db-438e-bdf0-9e684a115a76}]
                  DEBUG - AbstractMessageHandler.handleMessage(56) | org.springframework.integration.handler.LoggingHan dler@c943d1 received message: [Payload=org.springframework.integration.message.Me ssageDeliveryException: Dispatcher has no subscribers.][Headers={springintegration_timestamp=1253370628320 , springintegration_id=b1218469-e9db-438e-bdf0-9e684a115a76}]
                  ERROR - LoggingHandler.handleMessageInternal(83) | org.springframework.integration.message.MessageDel iveryException: Dispatcher has no subscribers.
                  at org.springframework.integration.dispatcher.Unicast ingDispatcher.doDispatch(UnicastingDispatcher.java :97)
                  at org.springframework.integration.dispatcher.Unicast ingDispatcher.dispatch(UnicastingDispatcher.java:9 0)
                  at org.springframework.integration.channel.AbstractSu bscribableChannel.doSend(AbstractSubscribableChann el.java:43)
                  at org.springframework.integration.channel.AbstractMe ssageChannel.send(AbstractMessageChannel.java:116)
                  at org.springframework.integration.channel.AbstractMe ssageChannel.send(AbstractMessageChannel.java:94)
                  at org.springframework.integration.channel.MessageCha nnelTemplate.doSend(MessageChannelTemplate.java:22 3)
                  at org.springframework.integration.channel.MessageCha nnelTemplate.send(MessageChannelTemplate.java:180)
                  at org.springframework.integration.endpoint.SourcePol lingChannelAdapter.doPoll(SourcePollingChannelAdap ter.java:78)
                  at org.springframework.integration.endpoint.AbstractP ollingEndpoint$Poller.innerPoll(AbstractPollingEnd point.java:232)
                  at org.springframework.integration.endpoint.AbstractP ollingEndpoint$Poller.poll(AbstractPollingEndpoint .java:216)
                  at org.springframework.integration.endpoint.AbstractP ollingEndpoint$Poller.run(AbstractPollingEndpoint. java:209)
                  at org.springframework.integration.scheduling.SimpleT askScheduler$ErrorHandlingRunnableWrapper.run(Simp leTaskScheduler.java:307)
                  at java.util.concurrent.Executors$RunnableAdapter.cal l(Executors.java:441)
                  at java.util.concurrent.FutureTask$Sync.innerRunAndRe set(FutureTask.java:317)
                  at java.util.concurrent.FutureTask.runAndReset(Future Task.java:150)
                  at org.springframework.integration.scheduling.SimpleT askScheduler$TriggeredTask.run(SimpleTaskScheduler .java:256)
                  at java.util.concurrent.ThreadPoolExecutor$Worker.run Task(ThreadPoolExecutor.java:885)
                  at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:907)
                  at java.lang.Thread.run(Thread.java:619)

                  DEBUG - AbstractMessageChannel$ChannelInterceptorList.post Send(169) | postSend (sent=true) on channel 'errorChannel', message: [Payload=org.springframework.integration.message.Me ssageDeliveryException: Dispatcher has no subscribers.][Headers={springintegration_timestamp=1253370628320 , springintegration_id=b1218469-e9db-438e-bdf0-9e684a115a76}]

                  Comment


                  • #10
                    Hi Iwein, I pulled the source into eclipse and debugged. getClient method on QueuedFTPClientPool is never called. I need to debug further....if any ideas come to mind, i'm open to suggestions...many thanx

                    Comment


                    • #11
                      Heh, you're sure your app context is started? I'm using mockftpserver for my unit tests (using a FakeFtpServer). That should log some NOOP and LIST calls at least.

                      Code:
                         @BeforeClass
                          public static void setupAndStartFtpServer() {
                              fakeFtpServer = new FakeFtpServer();
                              fakeFtpServer.setServerControlPort(8021);
                              fakeFtpServer.addUserAccount(new UserAccount("*****", "****",
                                      "/"));
                      
                              FileSystem fileSystem = new UnixFakeFileSystem();
                              fileSystem
                                      .add(new DirectoryEntry("*****"));
                              fileSystem.add(new FileEntry(
                                      "*****"));
                              fileSystem.add(new FileEntry(
                                      "*****"));
                              fakeFtpServer.setFileSystem(fileSystem);
                      
                              fakeFtpServer.start();
                          }
                      Last edited by iwein; Sep 19th, 2009, 01:52 PM. Reason: misleading name

                      Comment


                      • #12
                        Hi Iwein, I appreciate the response, thank u.
                        I basically setup ftpPro on my local pc. I can ftp and logon etc with the details I'm using in the SI app context cfg. But...still no go...
                        As I'm debugging thru the ftpAdapter code, it look as though the Spring and commons-net ftpclient isnt instantiated correctly and getClient is never called, so an attempt to logon etc is never actually executed. Somehow, something is causing some weirdness. Perhaps there is a dependancy thats wrong...or something

                        Can u try without the test mocks, to access a real ftp server? What version of commons-net are u using? Can you pls let me know what other dependancies you're using?

                        Comment


                        • #13
                          I don't have time to tune down my whole project to share it at the moment, but I'm absolutely sure that BUILD-27 of the project will work (also with a real ftp server).

                          Can you check if afterPropertiesSet() and start() are called on the synchronizer? I'm thinking that the synchronizer's taskscheduler is simply not running. I might be able to do something there to ensure that it will always be started, at the latest when receive is called the first time on the FtpFileSource.

                          Comment


                          • #14
                            Hi Iwein,

                            Really appreciate the pointers. I downloaded sources from...

                            https://build.springsource.org/brows...rget/artifacts

                            I will check afterPropertiesSet and start methods, just debug through the app again.
                            I'm determined to get this adapter working, will really save us a lot of time. I've built this entire spring integration app for a large project, so, invested a great deal of time to branch/break out into an alternative way of doing things. Would really love to keep doing things the SI way. Ok lemme, so take a look again....

                            Comment


                            • #15
                              Originally posted by iwein View Post
                              I don't have time to tune down my whole project to share it at the moment, but I'm absolutely sure that BUILD-27 of the project will work (also with a real ftp server).

                              Can you check if afterPropertiesSet() and start() are called on the synchronizer? I'm thinking that the synchronizer's taskscheduler is simply not running. I might be able to do something there to ensure that it will always be started, at the latest when receive is called the first time on the FtpFileSource.
                              Hi Iwein,

                              Yup, as u said, the synchronizer.start(); isnt called, yet the afterPropertiesSet() method is called. Anything I can do to help, with the change u mentioned above? Would really mean a lot to me to have this working. I can re-contribute in some way I'm sure.

                              Comment

                              Working...
                              X