Announcement Announcement Module
Collapse
No announcement yet.
MultiThreading using dispatcher and task-executor Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • MultiThreading using dispatcher and task-executor

    Hi All,

    I have requirement where I need to have different jobs to generate, parse,load XML files in database, archive, purge it. I am using spring batch to define different jobs related to this one.

    To generate, parse and load XML files I am using spring integration which will be invoked from Tasklet to generate, parse and load XML files into Database. I am utilizing multi-threading concept to run all this activity in different thread for different XML files.

    I am using dispatcher and task-executor concept as mentioned below in spring integration configuration file to initiate the multiple threads to perform action.

    <channel id="channel1">
    <dispatcher task-executor="TaskExecutor"/>
    </channel>

    <task:executor id="TaskExecutor" pool-size="6"/>

    <service-activator id="generateService" input-channel="channel1" ref="generateXMLFiles" output-channel="channel2" method="execute">

    <channel id="channel2"></channel>


    Based on this configuration system is initiating multiple threads and running the service activator "generateService" in different threads but it is not attaching with main thread. Due to which main thread is not waiting for all the child-thread to complete and shutting down complete process which is making child thread also to terminate before it completes its job.

    Can some one help me with solution in case if I am doing some thing wrong here or if i am not using correct configuration to carry out the activity which I am intend to?

    Thanks in Advance.

    Regards,
    Nemish

  • #2
    This is the expected behavior with the executor channel. If you want the requesting thread to wait for the processing to be done then use a direct channel instead of executor channel (channel 2 in your config is a direct channel).
    Are you spawning more threads withing the service activator? If yes, making channel1 direct wont help too.

    Does that answer your question?

    Comment


    • #3
      Hi Amol,

      Thanks for your reply. Requirement for me is that, Channel1 will get 10 messages in channel. I need Threadpool which read messages from channel1 in different thread and carry out processing. Out put at the end of each thread should go to Channel 2.

      Can you through some light on how or whether I can achive this using spring integration framework or not?

      Regards,
      Nemish

      Originally posted by Amol Nayak View Post
      This is the expected behavior with the executor channel. If you want the requesting thread to wait for the processing to be done then use a direct channel instead of executor channel (channel 2 in your config is a direct channel).
      Are you spawning more threads withing the service activator? If yes, making channel1 direct wont help too.

      Does that answer your question?

      Comment


      • #4
        Are all 10 messages put in channel1 by the same thread or multiple threads are putting the messages? Where/how are you putting messages in the channel1?

        Comment


        • #5
          Hi Amol,

          I have ArrayList<String> on which I am using <splitter> to split it and put it in channel1. So to answer your question it comes from one thread.

          btw for my understanding how does it make difference whether it comes from different thread or one thread?

          Regards,
          Nemish

          Originally posted by Amol Nayak View Post
          Are all 10 messages put in channel1 by the same thread or multiple threads are putting the messages? Where/how are you putting messages in the channel1?

          Comment


          • #6
            If the request is coming in from 10 separate threads, you actually need not use the executor channel and a DirectChannel was fine. Now since you sender process iterating the list the sending and processing should happen in a separate thread.

            What environment you are executing this? As a junit? Your problem is seen as the main thread is sending the messages to the channel1 and then terminates.

            For junits i usually put a sleep long enough at the end to ensure the completion of the task (not an elegant way, but works for testing)

            Comment


            • #7
              No its not JUnit..... using this logic in real application.....


              Originally posted by Amol Nayak View Post
              If the request is coming in from 10 separate threads, you actually need not use the executor channel and a DirectChannel was fine. Now since you sender process iterating the list the sending and processing should happen in a separate thread.

              What environment you are executing this? As a junit? Your problem is seen as the main thread is sending the messages to the channel1 and then terminates.

              For junits i usually put a sleep long enough at the end to ensure the completion of the task (not an elegant way, but works for testing)

              Comment


              • #8
                Ok,
                Then have a class which acts as a container. On invoking start method of this class, your spring application context will get initialized and once your application context gets initialized you can do something like

                Code:
                Semaphore sem = new Semaphore(0);
                sem.acquire();
                or
                Code:
                Object o = new Object();
                synchronize(o) {
                 o.wait();
                }
                This will keep your main thread alive.

                Also have some kind of destroy method that will shutdown the context and application.



                Also do somethings like:
                1. You can register a shutdown hook with the application context to ensure graceful shutdown in case of an unforeseen termination
                2. in your destroy call application context's close method and do
                Code:
                o.notify()
                or
                Code:
                sem.release()
                as per what you have used in start method.

                Does that answer your query?

                Comment

                Working...
                X