Announcement Announcement Module
Collapse
No announcement yet.
JDBCMessageStore and MimeMailMessage Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • JDBCMessageStore and MimeMailMessage

    I have created a MailGateway for my application which uses a JDBCMessageStore to back to queue:

    Code:
    @Gateway
    public void sendMail(OutgoingMailMessage recipient);
    and configured it as follows:

    Code:
    <channel id="outboundMailChannel" >
      <queue message-store="messageStore" />
    </channel>
    <jdbc:message-store data-source="dataSource" id="messageStore" />
    This used to send SimpleMailMessage instances perfectly well. I have now found that to send HTML e-mail I need to use MimeMailMessage. However, I now get the following exception when I try to send an e-mail:

    Code:
    Failed to serialize object using DefaultSerializer; nested exception is java.io.NotSerializableException: org.springframework.mail.javamail.MimeMailMessage
    This makes sense because SimpleMailMessage is Serializable and MimeMailMessage is not. I'd like to know if anyone has any ideas to allow this to work? I can fix it for now by removing the JDBCMessageStore but am looking for something better!
    Last edited by alexbarnes; Mar 2nd, 2012, 04:32 AM. Reason: Fix typo

  • #2
    The first thing that comes to mind is that you can reference any implementation of the Spring Serializer interface via the 'serializer' attribute on the JDBC Message Store configuration element. That way, you can choose an alternative to the standard Java serialization (which is obviously the default). Of course, you'd need to implement the symmetric Deserailzer behavior.

    This is one area that is really calling out for extensions (implementations of those strategies). We hope to have several out-of-the-box options at some point, similar to the way Spring OXM supports so many different XML marshalling technologies. Of course, community contributions are more than welcome!

    Thanks,
    Mark

    Comment


    • #3
      You can provide a serializer and deserializer attributes to the message-store something like

      Code:
      <jdbc:message-store data-source="dataSource" id="messageStore" serializer="serializer"  deserializer="deserializer"/>
      serializer and deserializer are beans of class implementing the org.springframework.core.serializer.Serializer org.springframework.core.serializer.Deserializer interfaces respectively. You need to provide your custom implementation there
      Last edited by Amol Nayak; Mar 1st, 2012, 08:45 AM.

      Comment


      • #4
        Thanks both for your prompt replies. I did try replacing the default Serializer with my own implementation. However, the issue here is that the MimeMailMessage doesn't implement Serializable.

        This was my attempt. Note that it just circumvents the check for the Serializable interface!

        Code:
        public void serialize(Object object, OutputStream outputStream)
        			throws IOException {
          ObjectOutputStream objectOutputStream = new ObjectOutputStream(outputStream);
          objectOutputStream.writeObject(object);
          objectOutputStream.flush();
        }
        However, this doesn't work because the MimeMailMessage isn't Serializable. Do you have any suggestions for the actual implementation of my Serializer? I'm very happy to contribute the code back to the community if I write one and it's useful! Perhaps you could point me in the right direction...
        Last edited by alexbarnes; Mar 1st, 2012, 08:54 AM.

        Comment


        • #5
          That is exactly why we use serializer, you *donot* serialize the MimeMailMessage . This is painful but you need to extract the fields you are interested in like the from, to, bcc, subject, content etc and store it in a object that is serializable. On deserialization you recreate a MimeMailMessage based on these values stored.

          But as mark pointed out, this must indeed be a common scenario and we can have some serializer and deserializer supported out of the box. Feel free to raise a JIRA for this enhancement here

          Comment


          • #6
            Sounds simple enough. I'll get started. Thanks.

            Comment


            • #7
              Amol's right. What you showed above is basically the same as what you'd see in Spring's DefaultSerializer:

              Code:
              /**
               * Writes the source object to an output stream using Java Serialization.
               * The source object must implement {@link Serializable}.
               */
              public void serialize(Object object, OutputStream outputStream) throws IOException {
              	if (!(object instanceof Serializable)) {
              		throw new IllegalArgumentException(getClass().getSimpleName() + " requires a Serializable payload " +
              				"but received an object of type [" + object.getClass().getName() + "]");
              	}
              	ObjectOutputStream objectOutputStream = new ObjectOutputStream(outputStream);
              	objectOutputStream.writeObject(object);
              	objectOutputStream.flush();
              }
              ...and that's the source of the problem, since the object you pass is not Serializable.

              There is one other option. Rather than a Serializer, you could add something like object-to-json transformer upstream. I don't know off the top of my head if that would work with MMM out of the box, but it can be customized (its ObjectMapper) and that might end up being less work overall. Not sure if you're interested in that approach, but I figured I'd throw it out there.

              Other serialization options we would like to have in the future are protocol buffers, thrift, avro, etc.

              Comment


              • #8
                The DefaultSerializer is exactly where I got the code from for mine! I just removed the 'object instanceof Serializable' part to see what would happen. The result? Not much. Not suprising.

                Thanks for all your suggestions. I'll have a play with it later.

                P.S. Was very impressed with how easy it was to get the whole thing working with SimpleMailMessage. JDBC backed queue with polling etc.

                Comment


                • #9
                  Both,

                  I've had a go at implementing a Serializer and Deserializer for a MimeMailMessage. I am getting an e-mail delivered which is good news however, I am seeing odd behaviour in the logs. When I was using a simple mail message with the DefaultSerializer after my e-mail had been sent the TRACE logs went back to showing a poll finding no messages. I would guess that the messages were removed from the database once sent?

                  Now I am seeing a continuous stream of logs:

                  Code:
                  TRACE: org.springframework.integration.monitor.QueueChannelMetrics - Recording receive on channel(outboundMailChannel) 
                  TRACE: org.springframework.integration.channel.QueueChannel - preReceive on channel 'outboundMailChannel'
                  DEBUG: org.springframework.integration.channel.QueueChannel - postReceive on channel 'outboundMailChannel', message: [Payload=org.springframework.mail.javamail.MimeMailMessage@1577944][Headers={timestamp=1330642853688, id=89677c3a-297a-48e1-afd9-f173644c81b3, JdbcMessageStore.CREATED_DATE=1330642786346, JdbcMessageStore.SAVED=true}]
                  DEBUG: org.springframework.integration.endpoint.PollingConsumer - Poll resulted in Message: [Payload=org.springframework.mail.javamail.MimeMailMessage@1577944][Headers={timestamp=1330642853688, id=89677c3a-297a-48e1-afd9-f173644c81b3, JdbcMessageStore.CREATED_DATE=1330642786346, JdbcMessageStore.SAVED=true}]
                  TRACE: org.springframework.integration.monitor.SimpleMessageHandlerMetrics - messageHandler(org.springframework.integration.mail.MailSendingMessageHandler#0) message([Payload=org.springframework.mail.javamail.MimeMailMessage@1577944][Headers={timestamp=1330642853688, id=89677c3a-297a-48e1-afd9-f173644c81b3, JdbcMessageStore.CREATED_DATE=1330642786346, JdbcMessageStore.SAVED=true}]) :
                  DEBUG: org.springframework.integration.mail.MailSendingMessageHandler - org.springframework.integration.mail.MailSendingMessageHandler#0 received message: [Payload=org.springframework.mail.javamail.MimeMailMessage@1577944][Headers={timestamp=1330642853688, id=89677c3a-297a-48e1-afd9-f173644c81b3, JdbcMessageStore.CREATED_DATE=1330642786346, JdbcMessageStore.SAVED=true}]
                  DEBUG: org.springframework.integration.jdbc.JdbcMessageStore - Removing message from group with group key=9ea43c5e-5587-3754-ada7-99f7801195f2
                  DEBUG: org.springframework.integration.jdbc.JdbcMessageStore - Updating MessageGroup: 9ea43c5e-5587-3754-ada7-99f7801195f2
                  TRACE: org.springframework.integration.monitor.QueueChannelMetrics - Recording receive on channel(outboundMailChannel) 
                  TRACE: org.springframework.integration.channel.QueueChannel - preReceive on channel 'outboundMailChannel'
                  DEBUG: org.springframework.integration.jdbc.JdbcMessageStore - Updating MessageGroup: 9ea43c5e-5587-3754-ada7-99f7801195f2
                  DEBUG: org.springframework.integration.channel.QueueChannel - postReceive on channel 'outboundMailChannel', message: [Payload=org.springframework.mail.javamail.MimeMailMessage@1577944][Headers={timestamp=1330642853715, id=0ace5a05-fc81-49d9-b55c-8e86b904475b, JdbcMessageStore.CREATED_DATE=1330642786346, JdbcMessageStore.SAVED=true}]
                  DEBUG: org.springframework.integration.endpoint.PollingConsumer - Poll resulted in Message: [Payload=org.springframework.mail.javamail.MimeMailMessage@1577944][Headers={timestamp=1330642853715, id=0ace5a05-fc81-49d9-b55c-8e86b904475b, JdbcMessageStore.CREATED_DATE=1330642786346, JdbcMessageStore.SAVED=true}]
                  TRACE: org.springframework.integration.monitor.SimpleMessageHandlerMetrics - messageHandler(org.springframework.integration.mail.MailSendingMessageHandler#0) message([Payload=org.springframework.mail.javamail.MimeMailMessage@1577944][Headers={timestamp=1330642853715, id=0ace5a05-fc81-49d9-b55c-8e86b904475b, JdbcMessageStore.CREATED_DATE=1330642786346, JdbcMessageStore.SAVED=true}]) :
                  DEBUG: org.springframework.integration.mail.MailSendingMessageHandler - org.springframework.integration.mail.MailSendingMessageHandler#0 received message: [Payload=org.springframework.mail.javamail.MimeMailMessage@1577944][Headers={timestamp=1330642853715, id=0ace5a05-fc81-49d9-b55c-8e86b904475b, JdbcMessageStore.CREATED_DATE=1330642786346, JdbcMessageStore.SAVED=true}]
                  DEBUG: org.springframework.integration.jdbc.JdbcMessageStore - Removing message from group with group key=9ea43c5e-5587-3754-ada7-99f7801195f2
                  DEBUG: org.springframework.integration.jdbc.JdbcMessageStore - Updating MessageGroup: 9ea43c5e-5587-3754-ada7-99f7801195f2
                  DEBUG: org.springframework.integration.jdbc.JdbcMessageStore - Updating MessageGroup: 9ea43c5e-5587-3754-ada7-99f7801195f2
                  My GenericMessage headers are the same in my Serializer and Deserializer suggesting that they are correctly serialized and deserialized. My headers before and after are as follows:

                  Code:
                  {timestamp=1330642786346, id=62452ca5-d827-4ef3-8d72-3f0e761541ea, JdbcMessageStore.CREATED_DATE=1330642786346, JdbcMessageStore.SAVED=true}
                  {timestamp=1330642786346, id=62452ca5-d827-4ef3-8d72-3f0e761541ea, JdbcMessageStore.CREATED_DATE=1330642786346, JdbcMessageStore.SAVED=true}
                  Does anything obvious spring to mind?

                  My Serializer is as follows:

                  Code:
                  public void serialize(GenericMessage<MimeMailMessage> object,
                  			OutputStream outputStream) throws IOException {MimeMailMessageStore store = new     MimeMailMessageStore("[email protected]", "Test", "Test",object.getHeaders());
                  
                  // -- Serialize the storage object
                  ObjectOutputStream objectOutputStream = new ObjectOutputStream(outputStream);
                  objectOutputStream.writeObject(store);
                  objectOutputStream.flush();
                  Where the MimeMailMessageStore is an immutable class which implements Serializable and stores the recipient address, subject, content and message headers.

                  My deserializer is as follows:

                  Code:
                  @Inject
                  	private MimeMailMessage mailMessage;
                  
                  	public GenericMessage<MimeMailMessage> deserialize(InputStream inputStream)
                  			throws IOException {
                  		ObjectInputStream objectInputStream = new ObjectInputStream(inputStream);
                  		
                  		/*// -- Attempt to recreate our message
                  		try {
                  			mailMessage.getMimeMessageHelper().setTo("[email protected]");
                  			mailMessage.getMimeMessageHelper().setSubject("Test");
                  			mailMessage.getMimeMessageHelper().setText("Text", true);
                  			
                  			MimeMailMessageStore store = (MimeMailMessageStore) objectInputStream.readObject();
                  			
                  			Map<String, Object> headers = new HashMap<String, Object>(store.getHeaders());
                  			headers.put(MessageHeaders.ID, store.getHeaders().get(MessageHeaders.ID));
                  			
                  			return new GenericMessage<MimeMailMessage>(
                  					mailMessage, headers);
                  		} catch (Exception e) {
                  			throw new RuntimeException(e);
                  		}
                  		
                  	}
                  where the MimeMailMessage is an injected prototype scoped bean.

                  Your thoughts and tips would be appreciated!

                  Comment


                  • #10
                    I have dug a bit more and now I a little confused:

                    I am creating a GenericMessage<MimeMailMessage> when I deserialize. I believe that I want to put the id in the header back to what it was before? Correct?

                    However, if I construct a GenericMessage passing in a Map<String, Object> containing the headers the GenericMessage constructor will use this Map to construct a new MessageHeaders. The constructor on MessageHeaders will set the ID value regardless. Therefore we can never set it despite this comment:

                    Code:
                    /**
                    	 * The key for the Message ID. This is an automatically generated UUID and
                    	 * should never be explicitly set in the header map <b>except</b> in the
                    	 * case of Message deserialization where the serialized Message's generated
                    	 * UUID is being restored.
                    	 */
                    	public static final String ID = "id";
                    Have I missed something here?

                    I am using version 2.1.0.RELEASE.

                    Comment


                    • #11
                      Yes, you cannot restore the ID this way; it will only be restored if the entire message is (de)serialized. That said, having a new ID won't hurt anything at all.

                      Another alternative is to simply provide transformers before and after the message store.

                      gateway->transformMMMToYourSerializable->JDBC-backed-channel->transformFromYourSerializableToMMM->toMailAdapter.

                      That way, you don't need any custom serialization.

                      Comment


                      • #12
                        That does sound like a good alternative, I'll give it a go.

                        I just want to confirm something:

                        Yes, you cannot restore the ID this way; it will only be restored if the entire message is (de)serialized. That said, having a new ID won't hurt anything at all.
                        If I put a message on my JDBC backed queue with the polling mail adaptor stopped I get a message in my database as expected. If I then start the outbound mail adaptor via the control channel the polling outbound mail adaptor finds my message over and over again. I would have expected it to remove it and I was expecting it to remove it using the id? Is that incorrect?

                        This is the SQL in the JDBCMessageStore:

                        Code:
                        DELETE_MESSAGE = "DELETE from %PREFIX%MESSAGE where MESSAGE_ID=? and REGION=?"
                        This is called by PollingConsumer#doPoll() as part of releasing a processed message. I would have thought that deserializing a message and setting a different id would cause issues? Specifically the above SQL will never delete the message from the int_message table and the PollingConsumer will keep finding it?

                        It seems that if you allow a hook for a custom (de)serializer then you need to allow this id to be set. That is certainly the impression I get from the comment on the field.
                        Last edited by alexbarnes; Mar 2nd, 2012, 06:18 AM.

                        Comment


                        • #13
                          Yes, you are correct; sorry - only Java serialization will work in this case and, of course, it doesn't work for you because MMM is not serializable.

                          My transformer pair suggestion should work ok.

                          Comment


                          • #14
                            In terms of supplying custom (de)Serializers would you agree that you need to be able to set the id on a new message? Given that you'll have serialized the headers and will be deserializing them and setting them on your new message?

                            Comment


                            • #15
                              Further to this: I've cloned the spring integration repo and updated the MessageHeaders constructor. I can now serialize to db then start the polling channel adaptor using the control channel. This allows the deserialization code to set the id of the message to that of the message in the db:

                              The following logs follow:

                              Code:
                              INFO : org.springframework.integration.endpoint.PollingConsumer - started outboundMailChannelAdaptor
                              DEBUG: org.springframework.integration.handler.ServiceActivatingHandler - handler 'ServiceActivator for [org.springframework.integration.handler.ExpressionCommandMessageProcessor@125cf56]' produced no reply for request Message: [Payload=@outboundMailChannelAdaptor.start()][Headers={timestamp=1330698789238, id=4ef870e1-2625-496a-81b9-285f8e4b6500}]
                              DEBUG: org.springframework.integration.channel.DirectChannel - postSend (sent=true) on channel 'controlChannel', message: [Payload=@outboundMailChannelAdaptor.start()][Headers={timestamp=1330698789238, id=4ef870e1-2625-496a-81b9-285f8e4b6500}]
                              TRACE: org.springframework.integration.monitor.DirectChannelMetrics - StopWatch 'controlChannel.send:execution': running time (millis) = 23; [] took 23 = 100%
                              TRACE: org.springframework.integration.monitor.QueueChannelMetrics - Recording receive on channel(outboundMailChannel) 
                              TRACE: org.springframework.integration.channel.QueueChannel - preReceive on channel 'outboundMailChannel'
                              Hibernate: select mailsender0_.id as id5_, mailsender0_.error as error5_, mailsender0_.errorTime as errorTime5_, mailsender0_.userName as userName5_, mailsender0_.version as version5_ from MailSendError mailsender0_ where mailsender0_.errorTime>=?
                              DEBUG: org.springframework.integration.jdbc.JdbcMessageStore - Removing message from group with group key=9ea43c5e-5587-3754-ada7-99f7801195f2
                              DEBUG: org.springframework.integration.jdbc.JdbcMessageStore - Updating MessageGroup: 9ea43c5e-5587-3754-ada7-99f7801195f2
                              DEBUG: org.springframework.integration.jdbc.JdbcMessageStore - Updating MessageGroup: 9ea43c5e-5587-3754-ada7-99f7801195f2
                              DEBUG: org.springframework.integration.channel.QueueChannel - postReceive on channel 'outboundMailChannel', message: [Payload=org.springframework.mail.javamail.MimeMailMessage@1f2139c][Headers={id=abeb7a40-0e85-4811-ab5a-c5088a2fe3a9, timestamp=1330698789373, JdbcMessageStore.CREATED_DATE=1330697565516, JdbcMessageStore.SAVED=true}]
                              DEBUG: org.springframework.integration.endpoint.PollingConsumer - Poll resulted in Message: [Payload=org.springframework.mail.javamail.MimeMailMessage@1f2139c][Headers={id=abeb7a40-0e85-4811-ab5a-c5088a2fe3a9, timestamp=1330698789373, JdbcMessageStore.CREATED_DATE=1330697565516, JdbcMessageStore.SAVED=true}]
                              TRACE: org.springframework.integration.monitor.SimpleMessageHandlerMetrics - messageHandler(org.springframework.integration.mail.MailSendingMessageHandler#0) message([Payload=org.springframework.mail.javamail.MimeMailMessage@1f2139c][Headers={id=abeb7a40-0e85-4811-ab5a-c5088a2fe3a9, timestamp=1330698789373, JdbcMessageStore.CREATED_DATE=1330697565516, JdbcMessageStore.SAVED=true}]) :
                              DEBUG: org.springframework.integration.mail.MailSendingMessageHandler - org.springframework.integration.mail.MailSendingMessageHandler#0 received message: [Payload=org.springframework.mail.javamail.MimeMailMessage@1f2139c][Headers={id=abeb7a40-0e85-4811-ab5a-c5088a2fe3a9, timestamp=1330698789373, JdbcMessageStore.CREATED_DATE=1330697565516, JdbcMessageStore.SAVED=true}]
                              TRACE: org.springframework.integration.monitor.QueueChannelMetrics - Recording receive on channel(outboundMailChannel) 
                              TRACE: org.springframework.integration.channel.QueueChannel - preReceive on channel 'outboundMailChannel'
                              TRACE: org.springframework.integration.monitor.QueueChannelMetrics - Recording receive on channel(outboundMailChannel) 
                              TRACE: org.springframework.integration.channel.QueueChannel - preReceive on channel 'outboundMailChannel'
                              DEBUG: org.springframework.integration.jdbc.JdbcMessageStore - Updating MessageGroup: 9ea43c5e-5587-3754-ada7-99f7801195f2
                              TRACE: org.springframework.integration.channel.QueueChannel - postReceive on channel 'outboundMailChannel', message is null
                              DEBUG: org.springframework.integration.endpoint.PollingConsumer - Poll resulted in Message: null
                              DEBUG: org.springframework.integration.endpoint.PollingConsumer - Received no Message during the poll, returning 'false'
                              This is the behaviour I saw with the SimpleMailMessage and what I was expecting i.e. once the Message is picked up it's deleted and the poll logs return to finding no message.

                              My change to MessageHeader was:

                              Code:
                              if (!this.headers.containsKey(ID)) {
                              			if (MessageHeaders.idGenerator == null){
                              				this.headers.put(ID, UUID.randomUUID());
                              			}
                              			else {
                              				this.headers.put(ID, MessageHeaders.idGenerator.generateId());
                              			}
                              		}
                              Thoughts? I appreciate that not allowing the Id to be set is a deliberate design decision. But it seems to be at odds with allowing custom deserializers.

                              I prefer the serialization and deserialization approach to the multi transformer approach. This allows me to keep my arrangement as follows:

                              Gateway -> Direct Channel -> Transformer -> Queue Channel (JDBCMessageStore) -> Outbound Mail Channel Adaptor

                              In this setup the message was passed directly through the transformer and then the Channel Adaptor polled the queue channel looking for mail to send. I can then start and stop the outbound adaptor using a control channel.
                              Last edited by alexbarnes; Mar 2nd, 2012, 01:22 PM.

                              Comment

                              Working...
                              X