Announcement Announcement Module
Collapse
No announcement yet.
Strange SFTP error Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • Strange SFTP error

    Hi I am getting an exception

    failed to rename from /134560/blah_2.xml.writing to /134560/tv/blag_2.xml; nested exception is 4: Failure

    in the log I can see

    SftpSession:147 - Initial File rename failed, possibly because file already exists. Will attempt to delete file: /134560/blah_2.xml and execute rename again.
    2013-06-21 17:18:54 DEBUG SftpSession:153 - Delete file: /134560/blah_2.xml succeeded. Will attempt rename again

    so looks like should be working but it's not any ideas what might be happening here?

  • #2
    When a file is uploaded, it is (by default) written with a temporary name (appended with .writing by default). This is to prevent applications on the remote server finding a file that is partially transfrerred. When the transfer is complete the file is renamed.

    As you can see, there are two attempts to rename the file; the first failure is logged as "... possibly because the file ..."; if the delete is successful (or the file being present was not the problem) a second attempt it made. If it fails, the "failed to rename from ..." exception is thrown.

    The problem is on the server; it could be a permissions problem (the account doesn't have the rights to rename).

    I notice the temporary file is in a different directory to the final file (the final file is in a subdirectory. 'tv'); maybe the permissions are different, or maybe the 'tv' directory doesn't exist?

    However, the auto-creation of the directory (if enabled) creates the final directory so the fact your temporary directory is in the same tree, all should be well.

    I suggest you look at the logs on the server to see if any errors are logged there.
    Last edited by Gary Russell; Jun 22nd, 2013, 02:41 AM.

    Comment


    • #3
      Actually, I was mistaken in my original reply (I have now edited it). The final directory is auto-created so, as long as the temporary directory is in the same tree, all should be fine.

      Do you have auto-create-directory="true" ??

      Comment


      • #4
        The file with the .writing extension is in the same directory as the original file. it should be delting the file blah in the tv directory then renaming blah.writing to blah all in the same directory. BTW I do have auto-create-directory="true". The directory structure is like top/tv/2 where blah is being written to tv directory and several other xml files are being written to the 2 directory. All the directories are created be the sftp session. The files being uploaded to the 2 directory work every time but they are small files. The file uploaded to the tv directory fail if the file already exists. Not sure will keep investigating.

        Comment


        • #5
          I was basing my comments on what you posted
          failed to rename from /134560/blah_2.xml.writing to /134560/tv/blag_2.xml
          Please show your actual config (and the actual log, if possible).

          Comment


          • #6
            Hi Here is my config

            <bean id="sftpSessionFactory" class="org.springframework.integration.sftp.sessio n.DefaultSftpSessionFactory">
            <property name="host" value="somesftpserver"/>
            <property name="privateKey" value="file:/id_rsa"/>
            <property name="user" value="username"/>
            </bean>

            <bean id="cachingSessionFactory" class="org.springframework.integration.file.remote .session.CachingSessionFactory">
            <constructor-arg ref="sftpSessionFactory"/>
            <property name="poolSize" value="${sftp.connections}"/>
            <property name="sessionWaitTimeout" value="1000"/>
            </bean>

            <int:channel id="1OutputChannel" />
            <int:channel id="2OutputChannel" />
            <int:channel id="3OutputChannel" />


            <sftp:outbound-channel-adapter id="3ftpOutboundAdapter"
            session-factory="sftpSessionFactory"
            channel="3OutputChannel"
            charset="UTF-8"
            remote-directory-expression="'/tv/' + headers['test']"
            auto-create-directory="true" />

            <sftp:outbound-channel-adapter id="1SftpOutboundAdapter"
            session-factory="sftpSessionFactory"
            channel="1OutputChannel"
            charset="UTF-8"
            remote-directory-expression="'/tv/' + headers['test'] + '/' + headers['test2']"
            auto-create-directory="true"/>

            <sftp:outbound-channel-adapter id="2SftpOutboundAdapter"
            session-factory="sftpSessionFactory"
            channel="2OutputChannel"
            charset="UTF-8"
            remote-directory-expression="'/tv/' + headers['test']"
            auto-create-directory="true" />


            here is the log

            2013-06-24 09:56:48 DEBUG SimplePool:177 - Obtained org.springframework.integration.sftp.session.SftpS ession@78f84a65 from pool.
            2013-06-24 09:56:48 DEBUG SftpSession:147 - Initial File rename failed, possibly because file already exists. Will attempt to delete file: /tv/blah.xml and execute rename again.
            2013-06-24 09:56:49 DEBUG SftpSession:153 - Delete file: /tv/blah.xml succeeded. Will attempt rename again
            2013-06-24 09:56:49 DEBUG CachingSessionFactory:109 - Releasing Session back to the pool.
            2013-06-24 09:56:49 DEBUG SimplePool:210 - Releasing org.springframework.integration.sftp.session.SftpS ession@78f84a65 back to the pool
            2013-06-24 09:56:49 ERROR Main:204 - error processing test33.XLS
            org.springframework.integration.MessageDeliveryExc eption: Error handling message for file [/tmp/blah.xml.tmp]
            at org.springframework.integration.file.remote.handle r.FileTransferringMessageHandler.handleMessageInte rnal(FileTransferringMessageHandler.java:168)
            at org.springframework.integration.handler.AbstractMe ssageHandler.handleMessage(AbstractMessageHandler. java:73)
            at org.springframework.integration.dispatcher.Unicast ingDispatcher.doDispatch(UnicastingDispatcher.java :115)
            at org.springframework.integration.dispatcher.Unicast ingDispatcher.dispatch(UnicastingDispatcher.java:1 02)
            at org.springframework.integration.channel.AbstractSu bscribableChannel.doSend(AbstractSubscribableChann el.java:77)
            at org.springframework.integration.channel.AbstractMe ssageChannel.send(AbstractMessageChannel.java:157)
            at org.springframework.integration.channel.AbstractMe ssageChannel.send(AbstractMessageChannel.java:128)
            at Main.sendMessage(Main.java:217)
            at Main.processFile(Main.java:180)
            at Main.process(Main.java:123)
            at sun.reflect.NativeMethodAccessorImpl.invoke0(Nativ e Method)
            at sun.reflect.NativeMethodAccessorImpl.invoke(Native MethodAccessorImpl.java:57)
            at sun.reflect.DelegatingMethodAccessorImpl.invoke(De legatingMethodAccessorImpl.java:43)
            at java.lang.reflect.Method.invoke(Method.java:601)
            at org.springframework.expression.spel.support.Reflec tiveMethodExecutor.execute(ReflectiveMethodExecuto r.java:69)
            at org.springframework.expression.spel.ast.MethodRefe rence.getValueInternal(MethodReference.java:84)
            at org.springframework.expression.spel.ast.CompoundEx pression.getValueInternal(CompoundExpression.java: 57)
            at org.springframework.expression.spel.ast.SpelNodeIm pl.getTypedValue(SpelNodeImpl.java:102)
            at org.springframework.expression.spel.standard.SpelE xpression.getValue(SpelExpression.java:102)
            at org.springframework.integration.util.AbstractExpre ssionEvaluator.evaluateExpression(AbstractExpressi onEvaluator.java:126)
            at org.springframework.integration.util.MessagingMeth odInvokerHelper.processInternal(MessagingMethodInv okerHelper.java:227)
            at org.springframework.integration.util.MessagingMeth odInvokerHelper.process(MessagingMethodInvokerHelp er.java:127)
            at org.springframework.integration.handler.MethodInvo kingMessageProcessor.processMessage(MethodInvoking MessageProcessor.java:73)
            at org.springframework.integration.handler.ServiceAct ivatingHandler.handleRequestMessage(ServiceActivat ingHandler.java:67)
            at org.springframework.integration.handler.AbstractRe plyProducingMessageHandler.handleMessageInternal(A bstractReplyProducingMessageHandler.java:134)
            at org.springframework.integration.handler.AbstractMe ssageHandler.handleMessage(AbstractMessageHandler. java:73)
            at org.springframework.integration.dispatcher.Unicast ingDispatcher.doDispatch(UnicastingDispatcher.java :115)
            at org.springframework.integration.dispatcher.Unicast ingDispatcher.dispatch(UnicastingDispatcher.java:1 02)
            at org.springframework.integration.channel.AbstractSu bscribableChannel.doSend(AbstractSubscribableChann el.java:77)
            at org.springframework.integration.channel.AbstractMe ssageChannel.send(AbstractMessageChannel.java:157)
            at org.springframework.integration.channel.AbstractMe ssageChannel.send(AbstractMessageChannel.java:128)
            at org.springframework.integration.core.MessagingTemp late.doSend(MessagingTemplate.java:288)
            at org.springframework.integration.core.MessagingTemp late.send(MessagingTemplate.java:149)
            at org.springframework.integration.endpoint.SourcePol lingChannelAdapter.handleMessage(SourcePollingChan nelAdapter.java:97)
            at org.springframework.integration.endpoint.AbstractT ransactionSynchronizingPollingEndpoint.doPoll(Abst ractTransactionSynchronizingPollingEndpoint.java:8 2)
            at org.springframework.integration.endpoint.AbstractP ollingEndpoint$1.call(AbstractPollingEndpoint.java :146)
            at org.springframework.integration.endpoint.AbstractP ollingEndpoint$1.call(AbstractPollingEndpoint.java :144)
            at org.springframework.integration.endpoint.AbstractP ollingEndpoint$Poller$1.run(AbstractPollingEndpoin t.java:236)
            at org.springframework.integration.util.ErrorHandling TaskExecutor$1.run(ErrorHandlingTaskExecutor.java: 52)
            at org.springframework.core.task.SyncTaskExecutor.exe cute(SyncTaskExecutor.java:48)
            at org.springframework.integration.util.ErrorHandling TaskExecutor.execute(ErrorHandlingTaskExecutor.jav a:49)
            at org.springframework.integration.endpoint.AbstractP ollingEndpoint$Poller.run(AbstractPollingEndpoint. java:231)
            at org.springframework.scheduling.support.DelegatingE rrorHandlingRunnable.run(DelegatingErrorHandlingRu nnable.java:53)
            at org.springframework.scheduling.concurrent.Reschedu lingRunnable.run(ReschedulingRunnable.java:81)
            at java.util.concurrent.Executors$RunnableAdapter.cal l(Executors.java:471)
            at java.util.concurrent.FutureTask$Sync.innerRun(Futu reTask.java:334)
            at java.util.concurrent.FutureTask.run(FutureTask.jav a:166)
            at java.util.concurrent.ScheduledThreadPoolExecutor$S cheduledFutureTask.access$201(ScheduledThreadPoolE xecutor.java:178)
            at java.util.concurrent.ScheduledThreadPoolExecutor$S cheduledFutureTask.run(ScheduledThreadPoolExecutor .java:292)
            at java.util.concurrent.ThreadPoolExecutor.runWorker( ThreadPoolExecutor.java:1145)
            at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:615)
            at java.lang.Thread.run(Thread.java:722)
            Caused by: org.springframework.integration.MessagingException : Failed to write to '/tv/blah.xml.writing' while uploading the file
            at org.springframework.integration.file.remote.handle r.FileTransferringMessageHandler.sendFileToRemoteD irectory(FileTransferringMessageHandler.java:251)
            at org.springframework.integration.file.remote.handle r.FileTransferringMessageHandler.handleMessageInte rnal(FileTransferringMessageHandler.java:157)
            ... 51 more
            Caused by: org.springframework.core.NestedIOException: failed to rename from /tv/blah_2.zml.writing to /tv/blah.xml; nested exception is 4: Failure
            at org.springframework.integration.sftp.session.SftpS ession.rename(SftpSession.java:164)
            at org.springframework.integration.file.remote.sessio n.CachingSessionFactory$CachedSession.rename(Cachi ngSessionFactory.java:137)
            at org.springframework.integration.file.remote.handle r.FileTransferringMessageHandler.sendFileToRemoteD irectory(FileTransferringMessageHandler.java:247)
            ... 52 more
            Caused by: 4: Failure
            at com.jcraft.jsch.ChannelSftp.throwStatusError(Chann elSftp.java:2491)
            at com.jcraft.jsch.ChannelSftp.rename(ChannelSftp.jav a:1665)
            at org.springframework.integration.sftp.session.SftpS ession.rename(SftpSession.java:161)



            So let me know what you think.

            Comment


            • #7
              Why are there 3 outbound adapters?

              Something doesn't look right here....

              Failed to write to '/tv/blah.xml.writing'
              ...
              failed to rename from /tv/blah_2.zml.writing to /tv/blah.xml
              where does the "blah_2.zml" come from when you are sending "blah.xml" ??

              Comment


              • #8
                well blah_2.zml is a typo I didn't want to post all filenames and everything for the general public. It should say

                failed to rename from /tv/blah.xml.writing to /tv/blah.xml

                There are 3 outbound adapters because that's the only way I could get it to ftp to 3 different directories. Is there a better way to do it. Maybe that is the problem. I am not sure it's the first time I have used spring integration so there is a good chance I am not doing things the best way.

                I have 3 channels and I put a String into the channel I want to write to. So one channel goes to one sftp server so if I want to write to dir a I put a string in outbound channel that is connected to sftp outbound-adapter for directory a.


                thanks
                Jeff

                Comment


                • #9
                  Hi I fixed it I was able to just use one sftp output adapter and it works fine now. Sorry thanks for the responses though like I said I am new to Spring integration.

                  Comment

                  Working...
                  X