Announcement Announcement Module
Collapse
No announcement yet.
SI threading question Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • SI threading question

    Hi,

    I have a threading question that I'm not quite sure how to describe but I make an attempt here. Essentially I have a library for AppX which allows me to define an EventHandler that is called every time there is an event like a sale is made (fictional). (The protocol is TCP although there is an abstraction layer over it similar to the JMS MessageListener interface). AppX is not using SI but does have an executor thread pool of 32 threads. My app is AppY which has AppX as a dependency.

    I am thinking of declaring this event handler as an inbound channel adapter which then puts, let's say, SaleDetails pojo onto the next channel. After that there might be five other endpoints which do various other things. In this flow I am using only
    Code:
    <channel id="blah" />
    between endpoints which I understand to be a direct channel. It is basically a linear progression of endpoints connected by direct channels. I guess I could use a chain but so far it is not using a chain but that's another topic.

    So here's my question. When an event occurs the AppX event handler will be invoked on its own thread and then subsequently I am assuming that the remaining endpoints in the flow will also be invoked on the same AppX thread? If this is so then that particular AppX tcp related thread is tied up for longer than it needs to be. Is there any way to ensure that all flow activity after the inbound channel adapter happens on a separate thread.

    I know you can do do that using a pollable channel and then declaring a thread pool executor on the poller. However I am using direct channels for maximum throughput and low latency and would like to avoid polling and throttling. Another solution might be to actually use a manually created executor thread pool in the AppX event handler endpoint and then use a thread from that to do processing within that endpoint. However if there are any other architectural recommendations which will allow such thread decoupling. Essentially I am looking for a way to just enqueue onto the next channel and then relieve the event handler tcp related thread.

    Please let me know if anything in unclear and I'll do my best to explain further.

    Thanks.
    Last edited by Narada; Feb 8th, 2009, 05:21 AM.

  • #2
    Something like the below for the first channel in AppY may be suitable. You would then use direct channels for the rest of the processing in AppY. Although this is using a poller by defining a high value for receive-timeout and a low value for interval you can still achieve low latency. Where there is no work on the queue the threads will block waiting for messages to be put on the queue.


    Code:
    <channel id="inputQueue">
        <queue />
    </channel>
    			
    <thread-pool-task-executor id="executorPool" core-size="10"/>
    			
    <service-activator>
        <poller receive-timeout="60000" task-executor="executorPool">
            <interval-trigger interval="100"/>
        </poller>
    </service-activator>
    An alternative would be to look at the Rendezvous channel implementation Rendezvoud channel this will allow immediate hand off where a thread is available but will block threads from AppX where no thread in AppY is available to hand off to. This can be a nice way to do it since you are effectively throttling AppX where AppY is overloaded. You say you don't want to throttle but what do you want to happen where AppY can not keep up with the pace? The risk is that the work builds up in memory.
    Last edited by JonasPartner; Feb 8th, 2009, 02:39 PM. Reason: code format

    Comment


    • #3
      Thank you very much Jonas. Your suggestions are very interesting and helpful. I will try them out and see how they compare in testing.

      The sad thing is that at the time of development I have no idea of how concurrent it will need to be, what level of throughput we require and how frequently the event handler in AppX will be called as we simply don't have that information so I have to either work off my own preferences or develop for the worst case scenario without making too many assumptions.

      I am still pondering the pros and cons of each of the solutions you've provided and am not yet decided on which will be more appropriate. I guess after some testing and, if possible, knowing more about the performance requirements will help me decide. As I was saying on another thread the primary difficulty with EIP and SI after you've figured out what you want and how to achieve it is deciding on one of numerous ways of implementing that particular flow. :-)

      Which would you go for from the above two?
      Last edited by Narada; Feb 8th, 2009, 05:25 PM. Reason: Added question at end

      Comment


      • #4
        Originally posted by JonasPartner View Post
        An alternative would be to look at the Rendezvous channel implementation Rendezvous channel this will allow immediate hand off where a thread is available but will block threads from AppX where no thread in AppY is available to hand off to. This can be a nice way to do it since you are effectively throttling AppX where AppY is overloaded. You say you don't want to throttle but what do you want to happen where AppY can not keep up with the pace? The risk is that the work builds up in memory.
        For completeness I'll add that even with a QueueChannel throttling happens by default when the queue is full. From the docs: If the queue has reached capacity, then the sender will block until room is available.

        The QueueChannel will react better to changes in message flux, making it more scalable, the RendezVousChannel will perform better on a single message flow.
        Last edited by iwein; Feb 10th, 2009, 04:20 AM. Reason: typo

        Comment


        • #5
          With a queue channel configured as in my example you would get performance very close to the rendezvous channel so I would start with that and monitor the message count in the queue which exposes a getMesssageCount(). Probably the more interesting question would be the thread count. The max size for that would depend on limiting factors in AppY such as is it doing lots of blocking IO is it making use of resources such as DB connections where there is a known maximum. If there is no obvious limiting factor I would start with a two or three times the number of cores higher for more IO. Once you have the starting values then unfortunately you will have to play a bit.

          I can really see where you are coming from in terms of being somewhat overwhelmed by the possible options. Even assuming the structure of the steps is straight forward there are a lot of configuration options, timeouts, polling period, channel type, thread pools .... which will effect performance and capacity. We clearly have a multi-parameter optimization problem with no easy way to find an optimal or close to optimal solution. I think this is true both of SI and a large number of other frameworks taking this series of steps which can be executed asynchronously approach. I have been meaning for a while to see how amenable this would be to the application of a genetic algorithm. I think the input would be the steps and some configuration about the order of those steps. The GA could then be used to tune thread pools (both size and sharing between step execution), channel types, sync or async hand off, polling periods, ... for a given load. The solution would then be marked for fitness based on latency, throughput and not exceeding prescribed load averages+ refusals if any. I think this should be fairly easy to do for a single message flow in SI so will post back if I get some time to play soon. This would probably not help you since you don't really know what a representative load looks like so ideally you would like something that is self-tuning. An interesting area of research but applying that to SI would probably be a big job.

          Comment


          • #6
            Iwein thanks. Yes I was wondering about using bounded queues and that will prevent OOM too. I'll try that option first as both yourself and Jonas recommend.

            Jonas, thanks, what you say is absolutely fascinating. Although I'm not familiar with that area the applications of GA to automating and controlling the configuration of a flow globally based on heuristics and flow metadata are very intriguing indeed.

            On a vaguely parallel note some time back I suggested EIP flow visualisations via Spring IDE or Spring Tool Suite on another thread. This could be extended to using a graphical designer to construct and adjust flows which would then create the dsl and stub objects. Given endpoints it could also propose a variety of flows and flow configurations perhaps based on what you say. For something like this a visual tool seems just the thing. Alternatively I wonder how else it can be presented to the user.

            (I really hope the Spring related IDE support acquires Spring Integration support somewhat. I've spent the past two weeks creating numerous mock yet fully functional integration flows using XML DSL and stub objects (to be fleshed out later) and it would be very nice for this to be automated.)

            Comment

            Working...
            X