Announcement Announcement Module
Collapse
No announcement yet.
Throughput Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • Throughput

    Throughput is a v.important element in our system. Aside from increasing the channel limit, is there anything else which is critical in achieving our goal?

  • #2
    The receive timeout setting for a poller may also be important. For example, if you have a long receive timeout, the threads will be in a timed-wait state and able to respond immediately. On the contrary, with a short timeout and longer interval, the messages may sit in the channel longer. Of course, this is more about "responsiveness" than "throughput" per se, but when you are dealing with a lot of messages, they are of course related.

    Comment


    • #3
      Hi Mark,

      Here is the important part of our configuration.

      We are running JMeter which is configured to start 1000 threads(users) and get an update of sorts.

      Below is our starting point configuration. We have a huge server and plenty of memory. When pushing the channel up to 20 we see no improvement in response/thru-put. I suspect we are doing something wrong.

      Code:
      <poller id="defaultPoller" 
              default="true" 
              max-messages-per-poll="10">    
      
              <interval-trigger 
                  time-unit="SECONDS" 
                  initial-delay="5" 
                  interval="3" />
      </poller>
      
      <channel id="myChannel">
          <queue capacity="10" />                    
      </channel>
      
      .
      .
      .
      
      <aggregator 
              input-channel="..." 
              ref=".." 
              method="..."
              timeout="6000"
              send-partial-result-on-timeout="true"
              discard-channel="..."/>
      Given our example could you define what you mean by:

      * timeout setting for a poller and
      * max-messages-per-poll (what does this mean exactly, where does this operate in the bus)

      Further information
      Our first test has the aggregator timeout set very high to basically block the client.

      In further tests this will be v.low and further requests will be made to read the discard channel.

      Each JMeter thread should take around 5 seconds. I assume once the aggregator has received all its channel data it returns immediately?

      Update
      I now see what you mean by received-timeout property. Missed this one.

      Do you think the send-timeout will also be important here too?

      Thanks.
      Last edited by jamesclinton; Feb 9th, 2009, 12:41 PM. Reason: further information...

      Comment


      • #4
        you can set a task-executor property on the poller and reference a <thread-pool-task-executor/> there. This one you can tweak some more.

        Comment


        • #5
          Thought I'd add an update to this thread as it might help others.

          Our system under heavy load was poor with regard to throughput/response when using SI.

          We noticed through checking the logs that putting messages on the bus is generally very quick, but getting the results out was slow.

          When we added dedicated poller's and thread-pools to the service activators instead of using the default poller/scheduller the round trip time *greatly* increased.

          Correction

          [..] round trip time *greatly* decreased.
          Last edited by jamesclinton; Mar 11th, 2009, 08:36 AM.

          Comment


          • #6
            Thanks James. I think this would be a good candidate for an article or blog.

            -Mark

            Comment


            • #7
              Originally posted by jamesclinton View Post
              When we added dedicated poller's and thread-pools to the service activators instead of using the default poller/scheduller the round trip time *greatly* increased.
              Do you mean improved? If it really increased, then things degraded. ??

              Comment


              • #8
                Good point Greg! I didn't notice that myself

                James, you do mean that the *throughput* increased I hope?

                Comment


                • #9
                  oops sorry.

                  Yes, round trip time decreased.

                  Good spot.

                  Comment


                  • #10
                    Originally posted by jamesclinton View Post
                    When we added dedicated poller's and thread-pools to the service activators instead of using the default poller/scheduller the round trip time *greatly* increased^H^H^H^H^H^H^H^Hmproved.
                    With the default poller do you mean this?
                    Code:
                    	<poller id="poller" default="true" task-executor="pool">
                    		<interval-trigger interval="..."/>
                    	</poller>
                    	
                    	<thread-pool-task-executor id="pool" core-size="1" max-size="200"/>
                    I don't see how reusing the same executor for different endpoints would make a great difference with using a dedicated pool for consumption. If it does I feel we should repair that in the framework. I think it is overly complicated if you have to manage multiple threadpools in most scenarios.

                    Could you comment on that (and create an issue if it doesn't work well)?

                    Comment


                    • #11
                      Pool & Executor Per End Point

                      Here's a snapshot of what we basically have ended up with:

                      Code:
                         <poller id="defaultPoller" 
                              default="true" 
                              max-messages-per-poll="${esb.poller.maxMessagesPerPoll}"
                              receive-timeout="${esb.poller.receivedTimeout}"
                              send-timeout="${esb.poller.sendTimeout}"
                              task-executor="defaultPool">    
                           
                              <interval-trigger 
                                  time-unit="SECONDS" 
                                  initial-delay="${esb.poller.intervalTrigger.initialDelay}" 
                                  interval="${esb.poller.intervalTrigger.interval}" />
                              
                          </poller>   
                          
                          <thread-pool-task-executor id="defaultPool"  .. />
                              
                          <thread-pool-task-executor id="AA_RequestPool"     .. />
                              
                          <thread-pool-task-executor id="BB_RequestPool"      .. />
                         
                          <!-- channels omitted -->
                      
                          <service-activator input-channel="SOME_CHANNEL_AA"  .. >
                              <poller task-executor="AA_RequestPool" .. >
                              	<interval-trigger .. />
                              </poller>
                          </service-activator>
                          
                          <service-activator input-channel="SOME_CHANNEL_BB"  .. >
                              <poller task-executor="BB_RequestPool" .. >
                              	<interval-trigger .. />
                              </poller>
                          </service-activator>
                      Maybe this is wrong?
                      Last edited by jamesclinton; Mar 11th, 2009, 09:09 AM.

                      Comment


                      • #12
                        Thanks James for sharing that.

                        Obviously if it works for you it would be moronic for me to say that it's wrong. I would like it to work without the three thread pools. You should be able to get the same performance with a single thread pool imho. Since you have clearly given this a great deal of attention I'd like to see if perhaps Spring Integration is wrong. Did you reproduce your performance results in a standalone app we can test with?

                        Comment


                        • #13
                          Wouldn't be easy.

                          Best option is to probably alter your cafe example which our system is similar too.

                          I'll describe the configuration.

                          Code:
                          Direct Channel 
                                     -> Splitter 
                                                  -> Router 
                                                             -> Service Activator 
                                             <- Pub/Sub Channel 
                                <- Service Activator (simply forwards to) 
                          <-- Direct Channel
                          1. Execute REST call places the message on the bus.

                          2. Execute REST call to read messages from pollable channel matching message id.

                          I might be able to help you here, but it'll have to be next week.

                          Comment


                          • #14
                            I added the setup you describe to the sandbox. I don't really see how you'd do anything concurrently with only direct and pubsub channels. Where is your asynchronous handoff?

                            You described adding pollers to the service activators, so I'm guessing you have a few QueueChannels in between somewhere.

                            https://fisheye.springframework.org/...ation/?cs=2008

                            Comment


                            • #15
                              Originally posted by iwein View Post
                              ...I don't really see how you'd do anything concurrently with only direct and pubsub channels. Where is your asynchronous handoff?
                              Hopefully this will help?

                              Code:
                              Message<SocialInboxUpdateRequest> message = buildMessage(socialNetworkFeeds, user);
                                      
                              
                              MessageChannelTemplate channelTemplate = new MessageChannelTemplate(socialInboxUpdateRequestChannel);
                                      
                              boolean status = channelTemplate.send(message);
                              Originally posted by iwein View Post
                              ...You described adding pollers to the service activators, so I'm guessing you have a few QueueChannels in between somewhere.
                              My channel config.

                              Code:
                              <channel id="socialInboxUpdateRequest"/>
                              
                              <splitter
                                 ref="socialInboxUpdateRequestSplitter"
                                 input-channel="socialInboxUpdateRequest"
                                 method="split"
                                 output-channel="splitSocialInboxUpdateRequest">
                              </splitter>
                              
                              <channel id="splitSocialInboxUpdateRequest"/>
                                  
                              <router
                                ref="socialNetworkUpdateRequestRouter"
                                input-channel="splitSocialInboxUpdateRequest"
                                method="resolveUpdateRequestChannel">
                              </router>
                              
                              <channel id="SOCIAL_NETWORK_1">
                                 <queue capacity="${capacity}" />
                              </channel>                    
                              
                              <channel id="SOCIAL_NETWORK_N">
                                 <queue capacity="${capacity}" />                    
                              </channel>
                              The pub/sub channel collects output from the service activators, 1 of my consumers is a simple pojo which forwards the result to a pollable channel where it can be picked up.
                              Last edited by jamesclinton; Mar 13th, 2009, 09:26 AM.

                              Comment

                              Working...
                              X