Announcement Announcement Module
Collapse
No announcement yet.
Spring Integration and SMB file inbound channel Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • Spring Integration and SMB file inbound channel

    Hi folks,

    I'm currently working on an application involving several inbound channels like http, sftp, ftp, ...
    These protocols are very well implemented in Spring Integration

    What I need as well is an inbound channel for downloading data from an "smb://"-URL - including authentication etc. As far as I know I can use the JCIFS library for this task. But it would be much better to have it available in Spring Integration too.
    • Does anyone know how to access files via the smb-protocol using Spring Integration?
    • Has anyone used the JCIFS library in combination with the Spring framework?

    Any hints would be appreciated.

    Thanks in advance
    Gunnar

  • #2
    Hi there,

    The Spring Integration framework provides a very flexible foundation that you can build on to support any type of file system. Spring Integration supports, for example, SFTP, FTP(FTPS), and regular file system mounts. We're of course interested in a SMB implementation/CIFS, as well.

    To build your own, check out https://github.com/SpringSource/spri...n/sftp/inbound and https://github.com/SpringSource/spri...n/sftp/session for the two key peices you need to support an inbound file adapter (there's also an outbound pacakge, but to get started with the inbound stuff, let's just focus on these two packages.)

    All Spring Integration file adapters support the concept of an inbound message source (a MessageSource implementartion, which is the standard interface for inbound, polling adapters) and a Synchronizer. The synchronizer knows how to download files and stash them in a local folder, and vice versa. Basically, all adapters poll the remote system (the file-system specific equivalent of ls -la) and pull the files that are new into a local tmp folder. Once the files are downloaded, they're moved into the folder that the adapter specifies for the files to be downloaded to.

    I'm glossing over a lot of this, because I can. Most of the code is already provided by the generic base classes. There are a few things that we can't "genericize," for example how to ask a file system for the name of a file, or how to ask the file system if a remote entry is a file (as opposed to a link or a directory), so that stuff you need to provide in the Synchronizer (see https://github.com/SpringSource/spri...chronizer.java. )

    The last thing that's important is the file system specific notion of a session. There are two parts: a session factory, and a session. A session factory is what describes the connection and authentication part of the adapter. Of course, this support varies from adapter to adapter: SSL certificates? Username and password? custom port and host? All of that stuff (and more) is buried in the custom implementation of a SessionFactory. The sessoin provides the adapter specific view of the file system, e.g., how to provide mkdir (and mkdir -p), cp, mv, ls -la, etc. See https://github.com/SpringSource/spri...n/sftp/session for a working example of some of these classes. Together, these two classes are where most of the work is done to "teach" spring integration about a new file system that can be used by the existing MessageSource and Synchronizer base classes (with just a few slight overrides that you provide).

    Finally, to get all this working and tested, i'd recommend first looking at the unit tests (https://github.com/SpringSource/spri...izerTests.java) of the some fo the existing file system adapters to see how the classes collaborate and work together.

    I hope this helps, if you get stuck please don't hesitate to post back here. And, of course, if you feel like open sourcing your work, don't hesitate to let us know that, too ;-)

    Comment


    • #3
      Hi Josh,

      Very good introduction - thanks for your detailed information !

      Looks like I can give it try and integrate JCIFS - just need a time slot in my current project...
      I'll let you know if my work suceeds.

      Regards
      Gunnar

      Comment


      • #4
        Hi Josh,

        the solution using the JCIFS library is pretty easy. As I have no time to open source the whole adapter here are the two main classes:

        SmbSession.java
        Code:
        package org.springframework.integration.smb.session;
        
        import ...
        
        /**
         * Default SMB {@link Session} implementation.
         */
        class SmbSession implements Session {
        	private final static Log logger = LogFactory.getLog(SmbSession.class);
        	
        	private final NtlmPasswordAuthentication ntlmAuth;
        	
        	private boolean isOpen;
        	
        	/**
        	 * The default constructor.
        	 * @param ntlmAuth NTLM user credentials
        	 */
        	public SmbSession(NtlmPasswordAuthentication ntlmAuth) {
        		Assert.notNull(ntlmAuth, "ntlmAuth must not be null");
        		this.ntlmAuth = ntlmAuth;
        		this.isOpen = true;
        	}
        
        
        	/**
        	 * @see org.springframework.integration.file.remote.session.Session#remove(java.lang.String)
        	 */
        	public boolean remove(String path) throws IOException {
        		try {
        			SmbFile file = new SmbFile(path, ntlmAuth);
        			file.delete();
        		} catch( SmbAuthException ex ) {
        		    // handle authentication related issue here
        			isOpen = false;
        			throw new NestedIOException("Failed to remove resource " + path, ex);
        		} catch( SmbException ex ) {
        		    // any special SMB related exception handling
        			throw new NestedIOException("Failed to remove resource " + path, ex);
        		}
        		if (logger.isDebugEnabled()){
        			logger.debug("Successfully removed resource " + path);
        		}
        		return true;
        	}
        
        	/**
        	 * @see org.springframework.integration.file.remote.session.Session#list(java.lang.String)
        	 */
        	@SuppressWarnings("unchecked")
        	public SmbFile[] list(String path) throws IOException {
        		try {
        			SmbFile file = new SmbFile(path, ntlmAuth);
        			SmbFile[] list = file.listFiles();
        			if (list == null) {
        				throw new NestedIOException("Failed to list resource - unable to resolve " + path);
        			}
        			return list;
        		} catch( SmbAuthException ex ) {
        		    // handle authentication related issue here
        			isOpen = false;
        			throw new NestedIOException("Failed to list resource " + path, ex);
        		} catch( SmbException ex ) {
        		    // any special SMB related exception handling
        			throw new NestedIOException("Failed to list resource " + path, ex);
        		}
        	}
        
        	/**
        	 * @see org.springframework.integration.file.remote.session.Session#read(java.lang.String, java.io.OutputStream)
        	 */
        	public void read(String source, OutputStream os) throws IOException {
        		try {
        			SmbFile file = new SmbFile(source, ntlmAuth);
        			InputStream is  = file.getInputStream();
        			FileCopyUtils.copy(is, os);
        		} catch( SmbAuthException ex ) {
        		    // handle authentication related issue here
        			isOpen = false;
        			throw new NestedIOException("Failed to read resource " + source, ex);
        		} catch( SmbException ex ) {
        		    // any special SMB related exception handling
        			throw new NestedIOException("Failed to read resource " + source, ex);
        		}
        		if (logger.isDebugEnabled()){
        			logger.debug("Successfully read resource " + source);
        		}
        	}
        
        	/**
        	 * @see org.springframework.integration.file.remote.session.Session#write(java.io.InputStream, java.lang.String)
        	 */
        	public void write(InputStream inputStream, String destination) throws IOException {
        		try {
        			SmbFile file = new SmbFile(destination, ntlmAuth);
        			OutputStream os = file.getOutputStream();
        			FileCopyUtils.copy(inputStream, os);
        		} catch( SmbAuthException ex ) {
        		    // handle authentication related issue here
        			isOpen = false;
        			throw new NestedIOException("Failed to write resource " + destination, ex);
        		} catch( SmbException ex ) {
        		    // any special SMB related exception handling
        			throw new NestedIOException("Failed to write resource " + destination, ex);
        		}
        		if (logger.isDebugEnabled()){
        			logger.debug("Successfully written resource " + destination);
        		}	
        	}
        
        	/**
        	 * @see org.springframework.integration.file.remote.session.Session#rename(java.lang.String, java.lang.String)
        	 */
        	public void rename(String pathFrom, String pathTo) throws IOException {
        		try {
        			SmbFile source = new SmbFile(pathFrom, ntlmAuth);
        			SmbFile dest = new SmbFile(pathTo, ntlmAuth);
        			source.renameTo(dest);
        		} catch( SmbAuthException ex ) {
        		    // handle authentication related issue here
        			isOpen = false;
        			throw new NestedIOException("Failed to rename from " + pathFrom + " to " + pathTo, ex);
        		} catch( SmbException ex ) {
        		    // any special SMB related exception handling
        			throw new NestedIOException("Failed to rename from " + pathFrom + " to " + pathTo, ex);
        		}
        		if (logger.isDebugEnabled()){
        			logger.debug("Resource " + pathFrom + " was successfully renamed to " + pathTo);
        		}	
        	}
        
        	/**
        	 * @see org.springframework.integration.file.remote.session.Session#mkdir(java.lang.String)
        	 */
        	public void mkdir(String directory) throws IOException {
        		try {
        			SmbFile file = new SmbFile(directory, ntlmAuth);
        			file.mkdirs();
        		} catch( SmbAuthException ex ) {
        		    // handle authentication related issue here
        			isOpen = false;
        			throw new NestedIOException("Failed to create directory " + directory, ex);
        		} catch( SmbException ex ) {
        		    // any special SMB related exception handling
        			throw new NestedIOException("Failed to create directory " + directory, ex);
        		}
        		if (logger.isDebugEnabled()){
        			logger.debug("Successfully created " + directory);
        		}	
        	}
        
        	/**
        	 * @see org.springframework.integration.file.remote.session.Session#close()
        	 */
        	public void close() {
        		isOpen = false;
        	}
        
        	/**
        	 * @see org.springframework.integration.file.remote.session.Session#isOpen()
        	 */
        	public boolean isOpen() {
        		return isOpen;
        	}
        
        }
        DefaultSmbSessionFactory.java
        Code:
        package org.springframework.integration.smb.session;
        
        import ...
        
        /**
         * Factory for creating {@link SmbSession} instances.
         */
        public class DefaultSmbSessionFactory implements SessionFactory {
        	
        	private volatile String domain;
        
        	private volatile String user;
        
        	private volatile String password;
        
        	public void setDomain(String domain) {	
        		this.domain = domain;
        	}
        
        	public void setUser(String user) {
        		this.user = user;
        	}
        
        	public void setPassword(String password) {
        		this.password = password;
        	}
        
        	public Session getSession() {
        		Assert.notNull(this.domain, "domain must not be null");
        		Assert.hasText(this.user, "user must not be empty");
        		Assert.isTrue(StringUtils.hasText(this.password), "password is required");
        		try {
        			NtlmPasswordAuthentication ntlmAuth = new NtlmPasswordAuthentication(domain, user, password);
        			SmbSession smbSession = new SmbSession(ntlmAuth);
        			return smbSession;
        		}
        		catch (Exception e) {
        			throw new IllegalStateException("failed to create SMB Session", e);
        		}
        	}
        
        }
        The rest is straight forward. You might want to integrate it in future versions of Spring Integration.

        There is one thing remaining:
        • All the FTP/SFTP/HTTP/HTTPS adapters require a local directory for their inbound-channel-adapter, right?
        • Is it possible to download directly e.g. from FTP to another outbound channel, in my case SMB...?
        • Is there a solution to prevent an inbound channel from transferring a file already existing on another outbound channel?

        Regards
        Gunnar

        Comment


        • #5
          Has this implementation been integrated in spring-integration-smb SI extension?

          Comment


          • #6
            Hi,

            The code for the SMB implementation has been a separate user-contribution but the the underlying code is certainly similar. Also, JCIFS (http://jcifs.samba.org/) is used as well.

            Source Code:

            https://github.com/SpringSource/spri...ntegration-smb

            Sample:

            https://github.com/SpringSource/spri...er/samples/smb

            Cheers,

            Gunnar

            Comment


            • rajeshgheware
              rajeshgheware commented
              Editing a comment
              Hi Gunnar,

              This project still in BUILD-SNAPSHOT stage. Any idea on when its first release will be available? Let me know if this adapter needs contributor to push from build to release stage. I am ready to contribute.

              Rajesh Gheware
          Working...
          X