Announcement Announcement Module
Collapse
No announcement yet.
FTP file download using spring integration Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • FTP file download using spring integration

    aim is to download a file from ftp unix server to windows machine .
    all relevant information is passed on to connect to remote ftp server.
    dont see any file getting downloaded
    please advice if any mistake .


    {code}
    port=22
    host=ftpldn00
    userId=intranet\sysftpgss
    password=ftpaccess
    LOC_DIR=C:\Ftp_test
    REMOTE_DIR=/sysFTPGss/GssTest/Outgoing/DFC/



    <bean id="ftpClientFtry" class="org.springframework.integration.ftp.session .DefaultFtpSessionFactory" >
    <property name="host" value="${host}"/>
    <property name="port" value="${port}"/>
    <property name="username" value="${userId}"/>
    <property name="password" value="${password}"/>
    </bean>



    <int-ftp:inbound-channel-adapter id="ftpInbnd"
    channel="ftpChannel"
    local-directory ="${LOC_DIR}"
    remote-directory ="${REMOTE_DIR}"
    filename-pattern="*.txt"
    auto-create-local-directory="true"
    session-factory="ftpClientFtry">
    <int: poller fixed-rate="1000"/>
    </int-ftp:inbound-channel-adapter>


    <int:channel id="ftpChannel">
    <int:queue />
    </int:channel>

    ----------------------------------------

    AbstractApplicationContext basiccontext = new ClassPathXmlApplicationContext("/reporting-context.xml", ReportStartUp.class);

    PollableChannel ftpChannel = context.getBean("ftpChannel", PollableChannel.class);
    Message<List<?>> fileList = (Message<List<?>>) ftpChannel.receive();
    {code}

  • #2
    Please use [ code ]... [ /code ] tags here (no spaces in brackets), not {code}

    I suggest you turn on DEBUG logging and look at the logs. You should see what's happening. There are probably no files that match the criteria, or the files already exist locally.

    Comment


    • #3
      Thanks for the reply, Gary.


      Here is the exception which i am getting. Can you please help? files exist in the remote ftp server at given directory.

      2013-03-01 15:58:24,343 INFO [org.springframework.context.support.DefaultLifecyc leProcessor] - Starting beans in phase 2147483647
      2013-03-01 15:58:24,345 INFO [org.springframework.integration.endpoint.SourcePol lingChannelAdapter] - started ftpInbnd
      2013-03-01 15:58:24,347 INFO [***** IneligibleReportStartUp *****] - ftpChannel ftpChannel
      2013-03-01 15:58:25,031 ERROR [org.springframework.integration.handler.LoggingHan dler] - java.lang.IllegalStateException: failed to create FTPClient
      at org.springframework.integration.ftp.session.Abstra ctFtpSessionFactory.getSession(AbstractFtpSessionF actory.java:139)
      at org.springframework.integration.file.remote.sessio n.CachingSessionFactory.doGetSession(CachingSessio nFactory.java:100)
      at org.springframework.integration.file.remote.sessio n.CachingSessionFactory.getSession(CachingSessionF actory.java:79)
      at org.springframework.integration.file.remote.synchr onizer.AbstractInboundFileSynchronizer.synchronize ToLocalDirectory(AbstractInboundFileSynchronizer.j ava:143)
      at org.springframework.integration.file.remote.synchr onizer.AbstractInboundFileSynchronizingMessageSour ce.receive(AbstractInboundFileSynchronizingMessage Source.java:144)
      at org.springframework.integration.endpoint.SourcePol lingChannelAdapter.doPoll(SourcePollingChannelAdap ter.java:89)
      at org.springframework.integration.endpoint.AbstractP ollingEndpoint$1.call(AbstractPollingEndpoint.java :146)
      at org.springframework.integration.endpoint.AbstractP ollingEndpoint$1.call(AbstractPollingEndpoint.java :144)
      at org.springframework.integration.endpoint.AbstractP ollingEndpoint$Poller$1.run(AbstractPollingEndpoin t.java:207)
      at org.springframework.integration.util.ErrorHandling TaskExecutor$1.run(ErrorHandlingTaskExecutor.java: 52)
      at org.springframework.core.task.SyncTaskExecutor.exe cute(SyncTaskExecutor.java:48)
      at org.springframework.integration.util.ErrorHandling TaskExecutor.execute(ErrorHandlingTaskExecutor.jav a:49)
      at org.springframework.integration.endpoint.AbstractP ollingEndpoint$Poller.run(AbstractPollingEndpoint. java:202)
      at org.springframework.scheduling.support.DelegatingE rrorHandlingRunnable.run(DelegatingErrorHandlingRu nnable.java:53)
      at org.springframework.scheduling.concurrent.Reschedu lingRunnable.run(ReschedulingRunnable.java:81)
      at java.util.concurrent.Executors$RunnableAdapter.cal l(Executors.java:441)
      at java.util.concurrent.FutureTask$Sync.innerRun(Futu reTask.java:303)
      at java.util.concurrent.FutureTask.run(FutureTask.jav a:138)
      at java.util.concurrent.ScheduledThreadPoolExecutor$S cheduledFutureTask.access$301(ScheduledThreadPoolE xecutor.java:98)
      at java.util.concurrent.ScheduledThreadPoolExecutor$S cheduledFutureTask.run(ScheduledThreadPoolExecutor .java:207)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run Task(ThreadPoolExecutor.java:886)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:908)
      at java.lang.Thread.run(Thread.java:619)
      Caused by: java.net.ConnectException: Connection refused
      at java.net.PlainSocketImpl.socketConnect(Native Method)
      at java.net.PlainSocketImpl.doConnect(PlainSocketImpl .java:333)
      at java.net.PlainSocketImpl.connectToAddress(PlainSoc ketImpl.java:195)
      at java.net.PlainSocketImpl.connect(PlainSocketImpl.j ava:182)
      at java.net.SocksSocketImpl.connect(SocksSocketImpl.j ava:366)
      at java.net.Socket.connect(Socket.java:529)
      at org.apache.commons.net.SocketClient.connect(Socket Client.java:176)
      at org.springframework.integration.ftp.session.Abstra ctFtpSessionFactory.createClient(AbstractFtpSessio nFactory.java:152)
      at org.springframework.integration.ftp.session.Abstra ctFtpSessionFactory.getSession(AbstractFtpSessionF actory.java:136)
      ... 22 more

      Comment


      • #4
        Your configuration is incorrect...

        Caused by: java.net.ConnectException: Connection refused
        This means ${host} is not listening on ${port}

        Comment


        • #5
          Thanks a lot Gary. that helped alot and saved time.

          Can you please suggest a approach for following scenario?
          I get files from remote directory using FTP.

          Remote directory :
          tblCCLCounterparty_20130122_1536.txt
          tblCCLCounterparty_20130202_1530.txt
          tblCCLCounterparty_20130203_1236.txt

          how can i select latest file or say todays file ? Is there a way i can add a selector or how to build one ?

          my config so far is as follows :

          <bean id="ftpClientFtry" class="org.springframework.integration.ftp.session .DefaultFtpSessionFactory" >
          <property name="host" value="${host}"/>
          <property name="port" value="${port}"/>
          <property name="username" value="${userId}"/>
          <property name="password" value="${password}"/>
          </bean>


          <int-ftp:inbound-channel-adapter id="ftpInbnd"
          channel="ftpChannel"
          local-directory ="file:${LOC_DIR}"
          remote-directory ="${REMOTE_DIR}"
          filename-pattern="*.txt"
          auto-create-local-directory="true"
          session-factory="ftpClientFtry">
          <intoller fixed-rate="1000"/>
          </int-ftp:inbound-channel-adapter>

          please advice

          Comment


          • #6
            When posting code/config, please use [ code ] ... [ /code ] tags (no spaces in brackets).

            Yes, implement a FileListFilter (for type: FTPFile). You can then choose which file(s) to retrieve.

            Declare your filter as a <bean/> and provide a reference to it with the filter attribute on the adapter. You can't use a pattern as well as the filter so you'd have to add to code to select only *.txt files as well. You can do that in-line, or use a CompositeFileListFilter to wrap your filter and a FtpSimplePatternFileListFilter.

            Comment


            • #7
              thanks for all the help so far.

              Can you please help me with the following ?

              1. failed to send message to channel 'ftpChannel' within timeout: -1
              I see this message when file is downloaded using ftp . { ftp:inbound-channel-adapter } . the file gets downloaded , but i see above error. i am not sure what its trying to tell me and how do i avoid it ?

              2. how can i configure org.springframework.batch.item.file.FlatFileItemRe ader to look into the directory where ftp files are downloaded and start reading them for further processing.
              i have set resouce property to following
              Code:
               
              
              reading process starts with this . 
              
              <bean id="productReportReader" class="org.springframework.batch.item.file.FlatFileItemReader">
              		<property name="resource" value="file:C:\dev\workspace\eclipse 360\Report\Ftp_test\" />
              		<property name="linesToSkip" value="1" />
              		<property name="recordSeparatorPolicy" ref="productRecordSeparatorPolicy" />
              		<property name="lineMapper" ref="productLineMapper" />
              		<property name="strict" value="false" />
              	</bean>
              
              
              ftp process
              
              <bean id="ftpClientFtry"  class="org.springframework.integration.ftp.session.DefaultFtpSessionFactory" >
              		<property name="host" value="${host}"/>
              		<property name="port" value="${port}"/>
              		<property name="username" value="${userId}"/>
              		<property name="password" value="${password}"/>
              	</bean>
              
              <int-ftp:inbound-channel-adapter id="ftpInbnd"  
              		channel="ftpChannel"
              		local-directory ="${LOC_DIR}"
              		remote-directory ="${REMOTE_DIR}"
              		filename-pattern="*.txt"
              		auto-create-local-directory="true"
              		session-factory="ftpClientFtry">
              		<int:poller fixed-rate="1000"/>
              	</int-ftp:inbound-channel-adapter>

              Comment


              • #8
                You need to have something subscribed to channel 'ftpInbnd'.

                For launching Spring Batch jobs, take a look at the JobLaunchingMessageHandler in the Spring Batch Admin project.

                Comment


                • #9
                  with file list filter , i can select files with extention / dates etc . thanks for the suggestion.

                  Comment


                  • #10
                    Regarding , You need to have something subscribed to channel 'ftpInbnd'.
                    created a service-activator as follows. dummy class messagehandler has a few public methods.

                    Code:
                    <int-ftp:inbound-channel-adapter id="ftpInbnd"  
                    		channel="ftpChannel"
                    		local-directory ="${LOC_DIR}"
                    		remote-directory ="${REMOTE_DIR}"
                    		filter="fileFilter"
                    		auto-create-local-directory="true"
                    		session-factory="ftpClientFtry">
                    		  <int:poller fixed-rate="1000"/>  
                    	</int-ftp:inbound-channel-adapter>
                    
                    	<int:channel id="ftpChannel">
                    		<int:queue capacity="1000"/>
                    	</int:channel>
                    	
                    	<int:channel id="outChannel" >
                    		<int:queue capacity="1000"/>
                    	</int:channel>
                    
                    	<int:service-activator  input-channel="ftpChannel" output-channel="outChannel" ref="msgHandler" />
                    	<bean id="msgHandler" class="com.util.MessageHandler"/>
                    this gives error saying
                    org.springframework.beans.factory.BeanCreationExce ption: Error creating bean with name 'org.springframework.integration.config.ConsumerEn dpointFactoryBean#0': Invocation of init method failed; nested exception is java.lang.IllegalArgumentException: No poller has been defined for endpoint 'org.springframework.integration.config.ConsumerEn dpointFactoryBean#0', and no default poller is available within the context.

                    Caused by: java.lang.IllegalArgumentException: No poller has been defined for endpoint 'org.springframework.integration.config.ConsumerEn dpointFactoryBean#0', and no default poller is available within the context.

                    my aim to create service-activator was to get rid of
                    org.springframework.integration.MessageDeliveryExc eption: failed to send message to channel 'ftpChannel' within timeout: -1

                    please advice

                    Comment


                    • #11
                      Please browse through the samples (https://github.com/SpringSource/spri...ration-samples) and read the documentation http://static.springsource.org/sprin...eference/html/

                      When using a QueueChannel

                      Code:
                      	<int:channel id="ftpChannel">
                      		<int:queue capacity="1000"/>
                      	</int:channel>
                      The endpoint receiving messages from it needs a <poller/> child element.

                      But, you can simply remove the <queue/> element, and all will be well.

                      Comment

                      Working...
                      X