Announcement Announcement Module
Collapse
No announcement yet.
SFTP Adapter Channel not being closed Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • SFTP Adapter Channel not being closed

    I have the following Spring Configuration


    Code:
      <context:property-placeholder location="/spring/ftp/ftp.properties"/>
    
    	<bean id="sftpSessionFactory" class="org.springframework.integration.sftp.session.DefaultSftpSessionFactory">
    			<property name="host" value="${host}"></property>
    			<property  name="user" value="${username}"></property>
    			<property name="password" value="${password}"></property>
    	</bean>
    
      <int-sftp:inbound-channel-adapter id="ftpInbound"
      								  	channel="ftpChannel"
      								  	auto-startup="true"
      								  	session-factory="sftpSessionFactory"
                                        remote-directory="${sftp.remoteDir}"
                                        auto-create-local-directory="true"
                                        delete-remote-files="false"
                                        local-directory="file:output"
                                        filter="fileModifiedFilter">
        <int:poller fixed-rate="60000"/>
      </int-sftp:inbound-channel-adapter>
    
      <int:channel id="ftpChannel">
        <int:queue/>
      </int:channel>
    	<bean id="fileModifiedFilter" class="com.mycompany.AcceptModifiedFileFilter"></bean>

    Code:
    public class AcceptModifiedFileFilter<F> extends AbstractFileListFilter<F> {
    	/**
    	 * The logging object
    	 */
    	private static final Logger log = LoggerFactory.getLogger(AcceptModifiedFileFilter.class);
    
    	private final Map<String, Integer> fileModTime = new HashMap<String, Integer>();
    
    	private final Object monitor = new Object();
    
    	public AcceptModifiedFileFilter() {
    		// TODO Auto-generated constructor stub
    	}
    
    
    	public boolean accept(F file) {
    		
    		synchronized (this.monitor) {
    			if (this.fileModTime.get(file.toString())==null) {
    				fileModTime.put(file.toString(), ((com.jcraft.jsch.ChannelSftp.LsEntry) file).getAttrs().getMTime());
    				log.debug("First Instance of File" + " :: " + fileModTime  );
    				return true;
    			}
    			if (this.fileModTime.get(file.toString()).compareTo(((com.jcraft.jsch.ChannelSftp.LsEntry) file).getAttrs().getMTime()) == 0) {
    				log.debug("File already downloaded" + " :: " + fileModTime );
    				return false;
    				}
    			if (this.fileModTime.get(file.toString()).compareTo(((com.jcraft.jsch.ChannelSftp.LsEntry) file).getAttrs().getMTime()) < 0) {
    				// file was modified, update timestamp
    				log.debug("File Modified & new file is downloaded now" + " :: " + fileModTime);
    				fileModTime.put(file.toString(), ((com.jcraft.jsch.ChannelSftp.LsEntry) file).getAttrs().getMTime());
    				return true;
    				}
    				// modified in past
    			log.debug("File was modified in the past" + " :: " + fileModTime);
    				return false;	
    			}
    			
    		}
    }
    ---------------

    Implemented the Filter Solution -- now still I dont have the local files updated
    -------------------------------------Logoutput
    Code:
    02:41:11.274 [task-scheduler-2] DEBUG c.a.i.g.d.AcceptModifiedFileFilter - First Instance of File :: {drwxr-x---    2 hmsd     hmsd          256 Sep 22 13:19 .=1348319966}
    02:41:11.277 [task-scheduler-2] DEBUG c.a.i.g.d.AcceptModifiedFileFilter - First Instance of File :: {drwxr-x---    2 hmsd     hmsd          256 Sep 22 13:19 .=1348319966, drwxr-x---    6 hmsd     hmsd          256 Sep 22 09:49 ..=1348307398}
    02:41:11.277 [task-scheduler-2] DEBUG c.a.i.g.d.AcceptModifiedFileFilter - First Instance of File :: {drwxr-x---    2 hmsd     hmsd          256 Sep 22 13:19 .=1348319966, drwxr-x---    6 hmsd     hmsd          256 Sep 22 09:49 ..=1348307398, -rw-r-----    1 hmsd     hmsd      3928797 Sep 22 20:24 hierarchy_reln_flattened.csv=1348345449}
    02:41:11.278 [task-scheduler-2] DEBUG c.a.i.g.d.AcceptModifiedFileFilter - First Instance of File :: {drwxr-x---    2 hmsd     hmsd          256 Sep 22 13:19 .=1348319966, drwxr-x---    6 hmsd     hmsd          256 Sep 22 09:49 ..=1348307398, -rw-r-----    1 hmsd     hmsd      3293696 Sep 22 12:25 hierarchy_reln_flattened.xls=1348316748, -rw-r-----    1 hmsd     hmsd      3928797 Sep 22 20:24 hierarchy_reln_flattened.csv=1348345449}
    02:41:11.278 [task-scheduler-2] DEBUG c.a.i.g.d.AcceptModifiedFileFilter - First Instance of File :: {drwxr-x---    2 hmsd     hmsd          256 Sep 22 13:19 .=1348319966, drwxr-x---    6 hmsd     hmsd          256 Sep 22 09:49 ..=1348307398, -rw-r-----    1 hmsd     hmsd      3293696 Sep 22 12:25 hierarchy_reln_flattened.xls=1348316748, -rw-r-----    1 hmsd     hmsd         6039 Sep 22 20:42 sample.txt=1348346534, -rw-r-----    1 hmsd     hmsd      3928797 Sep 22 20:24 hierarchy_reln_flattened.csv=1348345449}
    02:43:05.240 [task-scheduler-2] DEBUG c.a.i.g.d.AcceptModifiedFileFilter - First Instance of File :: {-rw-r-----    1 hmsd     hmsd         6039 Sep 22 21:12 sample.txt=1348348368, drwxr-x---    2 hmsd     hmsd          256 Sep 22 13:19 .=1348319966, drwxr-x---    6 hmsd     hmsd          256 Sep 22 09:49 ..=1348307398, -rw-r-----    1 hmsd     hmsd      3293696 Sep 22 12:25 hierarchy_reln_flattened.xls=1348316748, -rw-r-----    1 hmsd     hmsd         6039 Sep 22 20:42 sample.txt=1348346534, -rw-r-----    1 hmsd     hmsd      3928797 Sep 22 20:24 hierarchy_reln_flattened.csv=1348345449}
    02:44:04.915 [task-scheduler-2] DEBUG c.a.i.g.d.AcceptModifiedFileFilter - First Instance of File :: {-rw-r-----    1 hmsd     hmsd         6039 Sep 22 21:12 sample.txt=1348348368, drwxr-x---    2 hmsd     hmsd          256 Sep 22 13:19 .=1348319966, drwxr-x---    2 hmsd     hmsd          256 Sep 22 21:14 .=1348348442, drwxr-x---    6 hmsd     hmsd          256 Sep 22 09:49 ..=1348307398, -rw-r-----    1 hmsd     hmsd      3293696 Sep 22 12:25 hierarchy_reln_flattened.xls=1348316748, -rw-r-----    1 hmsd     hmsd         6039 Sep 22 20:42 sample.txt=1348346534, -rw-r-----    1 hmsd     hmsd      3928797 Sep 22 20:24 hierarchy_reln_flattened.csv=1348345449}
    02:45:07.401 [task-scheduler-7] DEBUG c.a.i.g.d.AcceptModifiedFileFilter - File already downloaded :: {-rw-r-----    1 hmsd     hmsd         6039 Sep 22 21:12 sample.txt=1348348368, drwxr-x---    2 hmsd     hmsd          256 Sep 22 13:19 .=1348319966, drwxr-x---    2 hmsd     hmsd          256 Sep 22 21:14 .=1348348442, drwxr-x---    6 hmsd     hmsd          256 Sep 22 09:49 ..=1348307398, -rw-r-----    1 hmsd     hmsd      3293696 Sep 22 12:25 hierarchy_reln_flattened.xls=1348316748, -rw-r-----    1 hmsd     hmsd         6039 Sep 22 20:42 sample.txt=1348346534, -rw-r-----    1 hmsd     hmsd      3928797 Sep 22 20:24 hierarchy_reln_flattened.csv=1348345449}
    02:46:13.383 [task-scheduler-2] DEBUG c.a.i.g.d.AcceptModifiedFileFilter - First Instance of File :: {-rw-r-----    1 hmsd     hmsd         6039 Sep 22 21:12 sample.txt=1348348368, drwxr-x---    2 hmsd     hmsd          256 Sep 22 13:19 .=1348319966, drwxr-x---    2 hmsd     hmsd          256 Sep 22 21:14 .=1348348442, drwxr-x---    6 hmsd     hmsd          256 Sep 22 09:49 ..=1348307398, drwxr-x---    2 hmsd     hmsd          256 Sep 22 21:15 .=1348348533, -rw-r-----    1 hmsd     hmsd      3293696 Sep 22 12:25 hierarchy_reln_flattened.xls=1348316748, -rw-r-----    1 hmsd     hmsd         6039 Sep 22 20:42 sample.txt=1348346534, -rw-r-----    1 hmsd     hmsd      3928797 Sep 22 20:24 hierarchy_reln_flattened.csv=1348345449}
    Even if I see first Instance of the file the local repository is not being overwritten with the new file

    Is there any reason to this.
    Regards,
    Rams
    Last edited by ramalingam; Sep 22nd, 2012, 05:35 PM. Reason: Better Clarity of Question

  • #2
    The inbound adapter is not designed to do this; even if you get the synchronizer to work, the FileReadingMessageSource used to read the local directory has an AcceptOnceFileListFilter baked into it, which only uses the file name to decide whether or not a file has already been processed. So, replacing the file will not result in a new message.

    Consider using outbound gateways instead.

    http://static.springsource.org/sprin...tbound-gateway

    That way, you can 'ls' the remote directory, look at the file timestamp(s) and use 'get' to retrieve the file(s).

    There's an example showing how to use the gateway in the sample apps; you'd just need to add a filter to ignore files where the timestamp hasn't changed.

    Comment


    • #3
      Inbound Adaptors

      Thanx a lot it did help me to better understand what inbound adaptors are for.
      Originally posted by Gary Russell View Post
      The inbound adapter is not designed to do this; even if you get the synchronizer to work, the FileReadingMessageSource used to read the local directory has an AcceptOnceFileListFilter baked into it, which only uses the file name to decide whether or not a file has already been processed. So, replacing the file will not result in a new message.

      Consider using outbound gateways instead.

      http://static.springsource.org/sprin...tbound-gateway

      That way, you can 'ls' the remote directory, look at the file timestamp(s) and use 'get' to retrieve the file(s).

      There's an example showing how to use the gateway in the sample apps; you'd just need to add a filter to ignore files where the timestamp hasn't changed.

      Comment

      Working...
      X