Announcement Announcement Module
No announcement yet.
Making multithreaded flow single-threaded by using thread handover Page Title Module
Move Remove Collapse
Conversation Detail Module
  • Filter
  • Time
  • Show
Clear All
new posts

  • Making multithreaded flow single-threaded by using thread handover

    Hi guys,

    If my original event listener/handler is multithreaded - let's say a multithreaded consumer to a tcp broker of some sort - here's what I need to do.

    The event listener is receiving X messages at a time. However the remainder of my flow needs to be single threaded and process only one message at a time. What are the ways in which this can be reliably done?

    The ways I know are to use pull (poller), push with one subscriber (pubsub) and rendezvous channel. However how I can do this to fulfil following constraints - (1) messages should only be processed by one thread (2) the thread pool should never hit saturation and result in CALLER_RUNS at which point it effectively becomes multi-threaded. (3) messages should not be lost by the rejection policy.

    What would be the possible ways to achieve a thread hand over with the right hand side being single threaded without resulting in CALLER_RUNS or the message being lost or alternatively resulting in task rejected exceptions? Examples would be much appreciated.

    Many thanks.
    Last edited by Narada; Aug 21st, 2009, 06:22 AM.

  • #2
    What you need is a single-threaded Executor and a rejection policy other than CallerRuns. You would probably want pending tasks to accumulate in the Executor's queue. In the simplest case, you could have an unbounded queue, and then you would not need to worry about the rejection policy. *However*, that's pretty dangerous since it can lead to out-of-memory errors. The real question is then how do you want to handle excessive load? You could have a bounded queue and an "abort" policy so that the Exception becomes the caller's problem. Or, if you happen to have a situation where untimely messages are no longer important, you could use something like DiscardOldest (although you are probably not so lucky).

    Hopefully that is helpful. Maybe you can provide a bit more detail about how much load you are expecting, how inconsistent it may be, and how you want to deal with rejection?