Announcement Announcement Module
Collapse
No announcement yet.
Message received via POP3 and placed into RabbitMQ has empty Payload data Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • Message received via POP3 and placed into RabbitMQ has empty Payload data

    I am new to SI and after reading some examples, I am trying to write a program of my own that seems simple enough. The program attempts to read an email message via POP3 and put the message body on a RabbitMQ. It seemed to be working (there is a message on the queue) until I configured RabbitMQ Management so that I could view the Messages and much to my disappointment the payload is EMPTY!

    The server reported 1 messages remaining.
    Exchange si.test.exchange
    Routing Key si.test.binding
    Redelivered ●
    Properties
    priority: 0
    delivery_mode: 2
    headers:
    content_type: application/octet-stream
    Payload 0 bytes Encoding: string

    Please help as I have tried various variations (chains, different channels, etc) of the code with the same results.

    Here is my spring config:

    Code:
    <?xml version="1.0" encoding="UTF-8"?>
    <beans xmlns="http://www.springframework.org/schema/beans"
    	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    	xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
    		http://www.springframework.org/schema/integration http://www.springframework.org/schema/integration/spring-integration.xsd
    		http://www.springframework.org/schema/integration/mail http://www.springframework.org/schema/integration/mail/spring-integration-mail.xsd
    		http://www.springframework.org/schema/integration/file http://www.springframework.org/schema/integration/file/spring-integration-file-2.1.xsd
    		http://www.springframework.org/schema/util http://www.springframework.org/schema/util/spring-util.xsd
    		http://www.springframework.org/schema/integration/amqp http://www.springframework.org/schema/integration/amqp/spring-integration-amqp.xsd
    		http://www.springframework.org/schema/rabbit http://www.springframework.org/schema/rabbit/spring-rabbit-1.0.xsd" 
    	xmlns:int-amqp="http://www.springframework.org/schema/integration/amqp"
    	xmlns:int="http://www.springframework.org/schema/integration"
    	xmlns:int-mail="http://www.springframework.org/schema/integration/mail"
    	xmlns:util="http://www.springframework.org/schema/util"
    	xmlns:rabbit="http://www.springframework.org/schema/rabbit"
    	xmlns:si="http://www.springframework.org/schema/integration"
    	xmlns:file="http://www.springframework.org/schema/integration/file"
    	>
    
    	<int:channel id="incomingMail" >
        	</int:channel>
    
    	<int:channel id="outgoingRabbit" >
    	 <int:interceptors>
                <int:wire-tap channel="loggingChannel"/>
            </int:interceptors>
        	</int:channel>
    
    	<int:logging-channel-adapter id="loggingChannel" log-full-message="true" level="DEBUG"/>
    
    	<!-- replace 'userid and 'password' wit the real values -->
    	<int-mail:inbound-channel-adapter id="pop3ShouldDeleteTrue" 
    	                    	     store-uri="pop3://#########:[email protected]/INBOX" 
    			            channel="incomingMail" 
    			            should-delete-messages="true" 
    			            auto-startup="true" 
    			            java-mail-properties="javaMailProperties">
    		
    		<!-- Will poll every 20 seconds -->            
    		<int:poller fixed-rate="20000"/>
    			
    	</int-mail:inbound-channel-adapter>
    
    
    	<si:chain input-channel="incomingMail" output-channel="outgoingRabbit">
    		<si:transformer expression="payload.content"/>
    	</si:chain>
    
    
    	<int-amqp:outbound-channel-adapter channel="outgoingRabbit"
    		amqp-template="amqpTemplate" exchange-name="si.test.exchange"
    		routing-key="si.test.binding"/>
    
    
    	<util:properties id="javaMailProperties">
    		<prop key="mail.pop3.socketFactory.fallback">false</prop>
    		<prop key="mail.debug">true</prop>
    		<prop key="mail.pop3.port">995</prop>
    		<prop key="mail.pop3.socketFactory.class">javax.net.ssl.SSLSocketFactory</prop>
    		<prop key="mail.pop3.socketFactory.port">995</prop>
    	</util:properties>
    
    
    	<!-- Infrastructure -->
        	<rabbit:connection-factory id="connectionFactory" /> 
        	<rabbit:template id="amqpTemplate" connection-factory="connectionFactory" />
        	<rabbit:admin connection-factory="connectionFactory" />
        	<rabbit:queue name="si.test.queue" />
        	<rabbit:direct-exchange name="si.test.exchange">
            	<rabbit:bindings>
                		<rabbit:binding queue="si.test.queue" key="si.test.binding"/>
            	</rabbit:bindings>
        	</rabbit:direct-exchange>
    
    </beans>

  • #2
    The default converter (SimpleMessageConverter) will create an empty payload if the data is not byte[], String, or Serializable.

    I suspect your payload.content expression is not resulting in a String as you expected. Depending on how the original mail message is constructed, you may need to use getBodyPart(...) on the content and then do further transformation as necessary.

    If you run with debug logging, you should see the payload type coming out of the transformer (chain).

    Comment


    • #3
      See the JavaDoc for MimeMessage.getContent() for more information....

      Code:
      ...
           * Return the content as a Java object. The type of this
           * object is dependent on the content itself. For 
           * example, the native format of a "text/plain" content
           * is usually a String object. The native format for a "multipart"
           * message is always a Multipart subclass. For content types that are
           * unknown to the DataHandler system, an input stream is returned
           * as the content. <p>
           *
           * This implementation obtains the content from the DataHandler,
           * that is, it invokes <code>getDataHandler().getContent()</code>.
           * If the content is a Multipart or Message object and was created by
           * parsing a stream, the object is cached and returned in subsequent
           * calls so that modifications to the content will not be lost.
      ...

      Comment


      • #4
        MIMEMessage Transformer for Spring Integration POP3 to AMPQ

        Originally posted by Gary Russell View Post
        See the JavaDoc for MimeMessage.getContent() for more information....
        Right you are Gary, I ended up changing my <chain> to a <transformer> like this:

        Code:
          <int:transformer input-channel="incomingMail" output-channel="outgoingRabbit" ref="MailTransformer" method="transformPayload"/>
                <bean id="MailTransformer" class="org.mdh.pop3amqp.MailTransformer"/>
        I also added a MailTransformer class with a transformPayload method like this. It parses a multi part MIME or a plain/text message, and puts the payload on a String.

        Code:
        package org.mdh.pop3amqp;
        import javax.mail.internet.MimeMessage;
        import javax.mail.*;
        import javax.mail.internet.*;
        import org.springframework.integration.transformer.AbstractPayloadTransformer;
        
        public class MailTransformer extends AbstractPayloadTransformer<MimeMessage, String> {
                        @Override
                        protected String transformPayload(MimeMessage message) throws Exception {
                        StringBuilder ret = new StringBuilder(0);
                        try {
        
                                if(message.getContent() instanceof Multipart){
                                        Multipart multipart = (Multipart) message.getContent();
        
                                        for (int x = 0; x < multipart.getCount(); x++) {
                                                BodyPart bodyPart = multipart.getBodyPart(x);
                                                String disposition = bodyPart.getDisposition();
                                                ret.append(bodyPart.getContent());
                                        }
                                }
                                else{ //Assume its a String
                                        ret.append(message.getContent());
                                }
                        }
                        catch(Exception e){
                                return "Error Parsing MIME Message";
                        }
        
                        return ret.toString();
                }
        }
        It works, the queue now shows:

        The server reported 0 messages remaining.
        Exchange si.test.exchange
        Routing Key si.test.binding
        Redelivered ○
        Properties
        priority: 0
        delivery_mode: 2
        headers:
        content_encoding: UTF-8
        content_type: text/plain
        Payload 14 bytes Encoding: string

        test payload

        Comment


        • #5
          Transforming MIME Message Headers into a proper message for RabbitMQ.

          Now that I have this working, I need a strategy to to:

          1. Add the MIME Message Headers to the Message destined for the Rabbit Queue. I am not sure how to do this. I saw a AbstractHeaderTransformer() class http://static.springsource.org/sprin...ansformer.html but I am not sure how to use it with what I already have...

          2. Deal with attachments. The code above can be changed to identify attachments in the bodyPart like this:
          Code:
           
          String disposition = bodyPart.getDisposition();
                                                  if (disposition != null && (disposition.equals(BodyPart.ATTACHMENT))) {
                                                          DataHandler handler = bodyPart.getDataHandler();
                                                  }
                                                  else{
                                                          ret.append(bodyPart.getContent());
                                                  }
          The question is how should the attachment be sent to the RabbitMQ. What formats have you seen in an integration parttern that goes between Mail and RabbitMQ? I know XML and JSON are possible options. I am not sure how this would be implemented in the transformPayload method I created to parse out the message body. I guess I could format the XML string manually and base64encode the attachment. Any advise would be appreciated.

          Comment


          • #6
            You can map any headers you want, using the mapped-request-headers attribute on the amqp adapter, e.g...

            Code:
            mapped-request-headers="foo*"
            There's no standard way to map a Java MimeMessage to an AMQP message; you're pretty much on your own. If you come up with a reasonable general solution, feel free to submit it for consideration as an extension... https://github.com/SpringSource/spri...ion-extensions

            Comment


            • #7
              Originally posted by Gary Russell View Post
              There's no standard way to map a Java MimeMessage to an AMQP message; you're pretty much on your own. If you come up with a reasonable general solution, feel free to submit it for consideration as an extension...
              There are only 2 approaches I can see:

              1 extract the MIME message headers and copy them to AMQP headers. Extract the MIME payload and attachments and copy them to the AMQP payload. This is the approach I am working on right now

              2. copy the entire MIME message (headers and payload) into the AMQP message payload . Not sure how to do this one. Any ideas? Converting the MIME Message to a Byte array and then to a string might work...

              I dont mind submitting the results as an extension. I think the community needs a standard way of doing this for a simpler , better mail integration.

              Comment

              Working...
              X