Announcement Announcement Module
Collapse
No announcement yet.
Multithreaded Integration model Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • Multithreaded Integration model

    Hi,

    I'm new to spring. I would like to know if we can create a multithreaded integration model.

    I want to listen from a jms queue and update my database. I want the process to run in say 'N' number of threads.

    Is there any way to achieve this. Any help is appreciated.

    Thanks,
    Jey

  • #2
    Hi!

    Classic case just to use concurrent-consumers on the <int-jms:message-driven-channel-adapter/>.

    Is it what are you looking for?

    Artem

    Comment


    • #3
      Hi Cleric,

      Just testing my understanding, if i give concurrent-consumers="5" there will be 5 concurrent threads? So it will read 5 messages from the queue at a time is that right?

      Thanks,
      Jey

      Comment


      • #4
        Yes.
        Just read JavaDocs: http://static.springsource.org/sprin...Container.html
        and Manual: http://static.springsource.org/sprin.../html/jms.html

        Comment


        • #5
          Is there any risk of same message being processed twice?

          Comment


          • #6
            Is there any risk of same message being processed twice?
            If you mean case of concurrent-consumers, then no.
            Otherwise it depends of your transaction atomicity:
            If your get a Message and process it in the new transaction (e.g. DB), then your JSM transction will be roll backed, so, that Message will return to the queue and will be processed again on the next consumption.
            So, just do entire message process in the JTA transaction, including JMS & DB.

            Comment


            • #7
              I was able to do it. Thanks .

              But there were few concerns. with concurrent-consumers few messages are lost and few messages are read multiple times.

              I had 52 messages in my queue and set the concurrent-consumers="10" and finally i got only 33 distinct messages(19 messages lost) and i had 17 duplicate messages.

              Am i missing any other configuration?

              Comment


              • #8
                Am i missing any other configuration?
                Show it!
                19 messages lost ... i had 17 duplicate messages
                How do you identify it?

                Comment


                • #9
                  Originally posted by Cleric View Post
                  Show it!
                  Code:
                  <si-jms:message-driven-channel-adapter 
                   		id="tibcoInboundAdapter" destination="tibcoInboundSendDestination"  
                   		connection-factory="tibcoAuthenticationConnectionFactory" error-channel="comsExceptionChannel" acknowledge="transacted" 
                   		channel="newOrderXMLInput" concurrent-consumers="10"/>
                  this is my listener configuration.

                  Originally posted by Cleric View Post
                  How do you identify it?
                  I have a table in which i will track all IDs and each message will have a distinct ID.

                  Comment


                  • #10
                    What is tibcoInboundSendDestination and how about transactions?
                    From other side, show, please, how do you deliver Message to the DB? Your message-flow, please.

                    Comment


                    • #11
                      I suspect you have some thread-safety issues in your down stream components.

                      Comment


                      • #12
                        I will not be able to show you my entire code.

                        tibcoInboundSendDestination is the jms queue configuration bean of type org.springframework.jndi.JndiObjectFactoryBean .

                        From other side, show, please, how do you deliver Message to the DB? Your message-flow, please.
                        I have channeled the messages to a DAO layer, before which i will unmarshall the message, and write the message contents to a DB.

                        Comment


                        • #13
                          Originally posted by Gary Russell View Post
                          I suspect you have some thread-safety issues in your down stream components.
                          Is there any way to identify whether I have any thread-safety issues?

                          Comment


                          • #14
                            tibcoInboundSendDestination is the jms queue configuration bean
                            Is it really queue? May it be a topic ?
                            I have channeled the messages to a DAO layer
                            Are all your channels direct? Don't you use publish-subscribe?

                            I recommend to go through the DEBUG from JmsMessageDrivenEndpoint to the DB INSERT. And try to figure out what's going on.
                            First of all, just switch on DEBUG on org.springframework.integration category and look at logs around your IDs.

                            Comment


                            • #15
                              Is it really queue? May it be a topic ?
                              It is a physical queue residing in TIBCO.

                              Are all your channels direct? Don't you use publish-subscribe?
                              Yes all the channels are direct. what would publish-subscribe do?

                              First of all, just switch on DEBUG on org.springframework.integration category
                              How do u do that? Is there any logger component in spring? currently I have wiretapped the inbound channel and i will log all the messages in coming to the application in a table along with their IDs. I have given my earlier statistics based on the data in that LOGGING table

                              Comment

                              Working...
                              X