Invoking beans in parallel

[xmaniac]

Hi,

So we would like to retrieve data from different sources (WS, DB, files) and such. We have the beans working but we are unsure about invoking them in parallel. How can this be achieved? Quartz? Spring Batch? Manually creating threads better?

Any ideas are welcomed

Regards,

[Alarmnummer]

It depends on the situation and you have not provided much information about situation. So the answer is very general. If you want a better answer, you need to give more information.

If you need some kind of timing based event, you could have a look at Quarts, but the ScheduledExecutor from Java 5 also is a very good candidate (it removes the need from introducing another library).

If you only need to invoke beans in parallel, I would look at the Executor (and ThreadPoolExecutor) from Java 5. If you need to work with Java 1.4, you can also have a look at the TaskExecutor from Spring (but it also ties you to Spring again)

I would watch out with using Spring batch for just running things in parallel. It is too complex for that (and it also ties you in the springframework and this is something that should be prevent unless there is a good reason).

[edit]
The main trick is to remove threading as much as you can from components. Hooking up the scheduler in the application context, just calling some method on some bean, is the way to go in most cases. In this way the choice for a parallel execution environment becomes just a configuration detail (the best kind of detail in most cases).

[xmaniac]

If you need some kind of timing based event, you could have a look at Quarts, but the ScheduledExecutor from Java 5 also is a very good candidate (it removes the need from introducing another library).

If you only need to invoke beans in parallel, I would look at the Executor (and ThreadPoolExecutor) from Java 5. If you need to work with Java 1.4, you can also have a look at the TaskExecutor from Spring (but it also ties you to Spring again)

We are already tied to Spring so no problem there. The WorkManagerTaskExecutor may be what we need as we are deploying to WAS. We try to achieve one thing: Invoke a collection of beans (data retrievers from different sources) in parallel, waiting for all them to finish before proceeding with the rest of the logic. Ideally the same beans should be available to be invoked serially.

Regards

[Paul Newport]

We are already tied to Spring so no problem there. The WorkManagerTaskExecutor may be what we need as we are deploying to WAS. We try to achieve one thing: Invoke a collection of beans (data retrievers from different sources) in parallel, waiting for all them to finish before proceeding with the rest of the logic. Ideally the same beans should be available to be invoked serially.

Regards

I haven't looked at it, but Spring Integration may help you:

http://www.springframework.org/spring-integration

[Alarmnummer]

We are already tied to Spring so no problem there.

I recommend not getting tied to Spring (or any other Framework) as long as you can.

The WorkManagerTaskExecutor may be what we need as we are deploying to WAS. We try to achieve one thing: Invoke a collection of beans (data retrievers from different sources) in parallel, waiting for all them to finish before proceeding with the rest of the logic. Ideally the same beans should be available to be invoked serially.

With java.util.concurrent this is peanuts to do. There are tons of classes that can help you here: Futures/ExecutorService, CountdownLatches, CyclicBarriers, CompletionServices.

Are you using Java 5+ or Java 1.4 (or even older)

[xmaniac]

I recommend not getting tied to Spring (or any other Framework) as long as you can.

We are using Spring MVC, WebFlow & Security so stripping Spring wouldn't be that easy.

With java.util.concurrent this is peanuts to do. There are tons of classes that can help you here: Futures/ExecutorService, CountdownLatches, CyclicBarriers, CompletionServices.

Are you using Java 5+ or Java 1.4 (or even older)

Good news! Java 5 fortunately. Any documentation or samples available?

Regards

[Alarmnummer]

We are using Spring MVC, WebFlow & Security so stripping Spring wouldn't be that easy.

Only create a dependency if you need to. In this case it is not needed.

Good news! Java 5 fortunately. Any documentation or samples available?

Regards

ExecutorService executorService = new ...;

List<Runnable> tasks = ....;
List<Future> futures = new LinkedList<Futures>();
for(Runnable task: tasks){
    Future future = executorService.submit(t);
    futures.add(future);
}

for(Future future: futures){
    future.get();
}

println("all tasks have completed, the after complete logic can be executed now")

This is just an example to show how it works.. I would really have a look at the completionservice.

[xmaniac]

Thank you again. I just keep a doubt. Even if my beans implement the Runnable interface (easy), the ExecutorService will probably instantiate a new object (not using the prototype passed as argument) and it won't have the dependencies injected. I mean, the Runnables are not executed inside the Spring context but in separate threads or am I confused?

Regards

[Alarmnummer]

Thank you again. I just keep a doubt. Even if my beans implement the Runnable interface (easy),

Ieuww... The runnable can call your service.

Runnable r = new Runnable{
   void run(){
       yourFunkyService.funkyMethod();
   }
}

executor.execute(r);

The runnable is just an adapter.

the ExecutorService will probably instantiate a new object (not using the prototype passed as argument) and it won't have the dependencies injected. I mean, the Runnables are not executed inside the Spring context but in separate threads or am I confused?

The dependencies the runnable needs, can be injected in to your 'SocketService'. Since the Runnable's are just stupid adapters, they can be created inside the 'SocketService'. You could create a RunnableFactory (nothing Spring specific) and inject it into your SocketService, but imho this is not needed in most cases.