Announcement Announcement Module
Collapse
No announcement yet.
Sending a message to multiple endpoints with transforms. Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • Sending a message to multiple endpoints with transforms.

    I'm currently trying to send a message (with xml payload) to 3 endpoints (file, email and webservice). Both the email endpoint and webservice endpoint require a transform on the xml payload before sending.

    To implement this I'm using a RecipientListRouter to send the message from its incoming channel onto 3 other channels, then performing transforms on the message (using the new transform interface from trunk) on the email and webservice channels. However using this setup everything seems to be acting on the same message object producing unpredictable results as the payload gets transformed.

    How should I implement this? Do I need to use a splitter to 'clone' the message? Is there any support for this or examples of it? Or is there a problem with the new transformer implementation in trunk?

    Thanks

    Dave

  • #2
    This cloning should probably happen in the router - perhaps as a configurable property of any router that may send to multiple channels (and should probably be 'clone=true' by default). Currently the router is just sending the same instance to each of the channels. It seems that the transformer should not care as long as the cloning is happening at the routing step. Does that make sense for your use case?

    -Mark

    Comment


    • #3
      Hi Mark,

      Thanks for your reply - yes I think that's exactly what I need. It should be up to the router to handle it. Should I raise a JIRA for this?

      Dave

      Comment


      • #4
        Actually I've looked at what I'm trying to do and think I should be using a publish subscribe channel instead of a router.

        The three targets email, webservice and file would be subscribed to this channel (This is really just an example - we are looking at providing a customisable event publisher mechanism on our product where for a particular integration an event may need to be sent to multiple targets)

        However the same problem exists - publish-subscribe channels send the same message instance to each subscribed target handler. That means if any of those handlers need to do different transforms they will all be acting on the same object.

        There seems to be a generic issue with sending a message to multiple targets and using transforms?

        Dave

        Comment


        • #5
          Dave,

          Would you mind opening a JIRA issue for the addition of a "clone" flag for pub/sub channels and any multi-channel router?

          I've started to make the changes for the router - basically anytime a router resolves channels and the set is > 1, then it will consult the flag (which is 'true' by default). To "clone" it simply invokes the GenericMessage constructor that accepts the payload and header (we might want to provide an actual clone operation instead). In general, does that solution sound appropriate for your scenario?

          Thanks,
          Mark

          Comment


          • #6
            Mark,

            Yes I'll raise a JIRA for this. The approach you suggest should be what we need assuming the dispatcher for a publish / subscribe channel does a similar thing - (ie an extra attribute setting on the channel in the xml config ie publish-subscribe=true clone=true) and it defaults to true (if publish-subscribe).
            The use of the GenericMessage constructor is fine for our needs for now.

            In general Spring Integration seems to be shaping up well for what we need. We intend to make use of OXM marshalling also and hence a lot of use of XML payloads. I'll probably raise some other JIRAs for things like XSLT transformers etc.

            Thanks

            Dave

            Comment


            • #7
              I get the feeling that Messages should just be immutable. That would make this whole problem go away.

              How about putting this in the javadoc:
              "Like any handler a transformer should refrain from modifying the incoming message or its payload. If a transformed message needs to be output it is required to create a new message with the transformed payload."

              Methods should not have side effects and all that...

              Comment


              • #8
                Well, the whole point of having Transformers is to modify the payload of an existing Message, rather than swallow-a-message-and-produce-another-one which is the behaviour of a HandlerEndpoint. For example, one might need to preserve the message identity.

                What is incorrect here and David pointed out quite well, is that we allow the same Message instance to be processed concurrently by multiple endpoints - which does not happen for linear message processing, but if the messages start being processed in parallel (pub-sub, multiple-destination-routing) this might become a problem when the message is manipulated (not only via payload transformation, but for example by a ContentEnricher). In fact, sometimes it might be desirable to create a copy of the payload itself if the transformers will treat the payload as a value object and modify it directly rather than replacing it (i.e. when the @Transformer method returns the same object as the argument).

                David,
                Thanks for adding the Jira item.

                Cheers,
                Marius
                Last edited by mbogoevici; Jun 14th, 2008, 12:21 PM.

                Comment


                • #9
                  Originally posted by mbogoevici View Post
                  Well, the whole point of having Transformers is to modify the payload of an existing Message, rather than swallow-a-message-and-produce-another-one which is the behaviour of a HandlerEndpoint. For example, one might need to preserve the message identity.
                  Marius
                  I can see perfectly well the problem that Dave is pointing out would impact users if they don't work with immutable messages. What I don't see is a functional reason to preserve message identity. I don't want to have the whole 'equals()' vs. '==' OR immutable vs. mutable discussion again, but my point is that there is an easy way out: use immutable messages.

                  Imho a copy of the message is just as good as the message itself (and in fact the solution proposed to dave's problem implies the same). I still don't get it I guess...

                  Transformation: one thing goes in, another thing comes out. Sounds good to me.

                  Comment


                  • #10
                    Originally posted by iwein View Post

                    Transformation: one thing goes in, another thing comes out. Sounds good to me.
                    Iwein,

                    There are cases when the transformation occurs on the message itself (either payload or header). The typical example for that is Content Enricher. On another note, the identity of the processed message might be necessary to be preserved for a Claim Check based handling ...

                    The MessageTransformer interface has been designed for this mutable Message case, and that's why transform() does not return anything.

                    But I'm afraid we moved away from Dave's original issue: this can be solved relatively easy now, by introducing supplemental Endpoints before the e-mail and webservice Targets, which would take charge of the transformation in a MessageHandler - rather than using a MessageTransformer - so the original shared copy is not altered, but rather a new message is created every time. Of course, this should just give some relief until the cloning option will be available.


                    Cheers,
                    Marius
                    Last edited by mbogoevici; Jun 14th, 2008, 10:24 PM. Reason: Explaining a solution within the confines of existing functionality

                    Comment


                    • #11
                      David,

                      We do plan to include an OXM-based MessageTransformer implementation in M5. Feel free to watch, vote, or add any comments to this JIRA issue: http://jira.springframework.org/browse/INT-109

                      If you'd like to open another issue for XSLT-based transformation, please do.

                      Regards,
                      Mark

                      Comment


                      • #12
                        Before I go off topic again, is the solution summarized by Marius workable for you Dave?

                        Originally posted by mbogoevici View Post
                        There are cases when the transformation occurs on the message itself (either payload or header). The typical example for that is Content Enricher. On another note, the identity of the processed message might be necessary to be preserved for a Claim Check based handling ...
                        It is perfectly possible to store a key in the message, use that to retrieve the data during the claim and create a new message with the data in it. There is no reason as far as I can see to require message identity preservation.

                        Relying on identity would be a bad idea imho. The whole point is loose coupling and if you don't know the route of the message you can't be sure that identity is preserved. Some component might persist messages in queues or send them over the network for example.

                        I'm not saying that mutable messages should be illegal or something, my only point is that I can't see a good reason you would want to use them. This brings us back to the transformer interface.

                        The MessageTransformer interface has been designed for this mutable Message case, and that's why transform() does not return anything.
                        It would be no problem to allow both mutable messages an immutable messages if we would return the message from the transform() method. I think that the interface needs to be changed. It doesn't have any advantages to return void.

                        Anyway, maybe I'm just addicted to immutable objects...

                        Comment


                        • #13
                          Originally posted by iwein View Post
                          ]
                          There is no reason as far as I can see to require message identity preservation.
                          The reason for being able to preserve message identity is:

                          http://www.eaipatterns.com/MessageStore.html

                          implemented as

                          org.springframework.integration.message.MessageSto re.

                          Originally posted by iwein View Post
                          ]
                          It would be no problem to allow both mutable messages an immutable messages if we would return the message from the transform() method. I think that the interface needs to be changed. It doesn't have any advantages to return void.
                          OK, let's make sure we're on the same page here: first, by discussing the fundamental difference between a Transformer and a Handler: a Transformer modifies the existing message, whereas the Handler grabs the message and potentially returns a new one (it may not return anything, for that matter).

                          If we would return something from the transform() method, there'd be no difference with a MessageHandler (please compare the two interfaces). Now, that'd be just duplicating existing functionality, which would be nonsensical, right ? So, the difference must be there for a reason, which is allowing the users to mutate the messages. There's no talk here on what's preferable in any given situation, it is what the framework supports or not.

                          If you want to use a strict immutable message-based processing model, which, in particular, I very much agree with, just use MessageHandlers instead of MessageTransformers for your handling, that's the whole point. Please understand their different purpose and different functionality.


                          Regards,
                          Marius
                          Last edited by mbogoevici; Jun 15th, 2008, 11:41 AM. Reason: spelling

                          Comment


                          • #14
                            OK, let's make sure we're on the same page here: first, by discussing the fundamental difference between a Transformer and a Handler: a Transformer modifies the existing message, whereas the Handler grabs the message and potentially returns a new one (it may not return anything, for that matter).

                            If we would return something from the transform() method, there'd be no difference with a MessageHandler (please compare the two interfaces). Now, that'd be just duplicating existing functionality, which would be nonsensical, right ? So, the difference must be there for a reason, which is allowing the users to mutate the messages. There's no talk here on what's preferable in any given situation, it is what the framework supports or not.

                            If you want to use a strict immutable message-based processing model, which, in particular, I very much agree with, just use MessageHandlers instead of MessageTransformers for your handling, that's the whole point. Please understand their different purpose and different functionality.
                            I do understand, but I still have some arguments with it. In any case we can take it offline. There is a clear understanding of Dave's problem, a solution and a workaround. My further arguments are irrelevant to this thread.

                            Comment


                            • #15
                              Originally posted by iwein View Post
                              In any case we can take it offline. There is a clear understanding of Dave's problem, a solution and a workaround.
                              Well, we walked a bit out of the main topic, as I said before. But I hope that it helped in explaining the fundamental differences between MessageTransformer and MessageHandler, as implemented by M4. And of course, let's hope it helps us build a better M5 .

                              Comment

                              Working...
                              X