Announcement Announcement Module
No announcement yet.
Aggregation as a service(scatter gather), backed by AMQP Page Title Module
Move Remove Collapse
Conversation Detail Module
  • Filter
  • Time
  • Show
Clear All
new posts

  • Aggregation as a service(scatter gather), backed by AMQP

    With reference to my previous thread of having performance issues while handling group of large messages
    refer :

    I am thinking of changing my design to expose aggregation as a service, in my previous implementation, i was employing a splitter connected to amqp out bound adapter (backed by executor channel) to publish messages parallely to amqp Services Queue. And I was using an amqp inbound adapter connected to a aggregator and a router to receive the reply messages.This did not scale very well. In this design splitting,aggregation happened in client environment itself.

    Now i am thinking of making aggregation it self behind an amqp Q, i.e. my client code will always put aggregated requests in an AMQP queue (using amqp outboud gateway), 1 or more set of SI instance which will do the aggreated calls using scatter,gather pattern ( i.e take the message from aggreation request Q(amqp), split it, send it to approprate service Q (Amqp),wait for replies, aggregate the result, and publish it back to aggregation response Q (amqp q).

    Please with enough experience in messaging, amqp, EIP patterns, please let me know if this solution will scale.

    I am looking for handing multiple parallel request ( upto order of 1000), with typical payload sizes of 400b to 40k, and a group of 100 to 200 such payloads in each aggregated service call.

    Any insights will be greatly appreciated.

    Please refer to my other posts ,related to same area for a better background of what i am trying :

    Thanks Madhu