Announcement Announcement Module
No announcement yet.
Aggregator in a cluster Page Title Module
Move Remove Collapse
Conversation Detail Module
  • Filter
  • Time
  • Show
Clear All
new posts

  • Aggregator in a cluster

    There is good fit of the aggregator pattern in our design. However, we need to run in a clustered environment and hence we want to have one aggregation for a given correlation key across the nodes in the cluster. I think we can use a message store. But, I am not sure if the message store is cluster aware or in other words, will it maintain a single instance of the aggregated entities across the nodes in the cluster?
    Is there an option to implement such a distributed, cluster aware cache / store using Gemfire?

    Please let me know.

  • #2
    Gemfire would be a reasonable choice, and we have an implementation in the sandbox, but I'm not sure if it's been used for aggregation yet (it is definitely used for claim check). We have tested the JDBC store with multiple concurrent clients - it is a bit of a bottleneck, but that is the price you pay for needing the global synchronization. Any store implementation you use has to support the equivalent of SERIALIZABLE transaction isolation (and if you use JDBC that's all you need, or you might get away with READ_COMMITTED depending on the platform). If you want to use Gemfire I would check the Spring Gemfire docs to see if SERIALIZABLE is supoprted.


    • #3
      Is it possible NOT to replicate the state of aggregator in a cluster, but route the messages from end points to specific aggregator node in a cluster, like one dynamically created aggregator listening only messages with specific message selector(correlation ID could be used as message filter for JMS channel). At the time splitter splits the messages, it should spawn an aggregator (prototype) that is configured to pick up only to aggregate certain messages with given correlation key. Can you please share your comments on this.


      • #4
        Rather than spawning a new aggregator for each group you can add a node-specific correlation header before the splitter (e.g nodeid + incrementingAtomicInteger) and then use a selector on the channel to only get messages for that nodeId.

        You would need a custom MessageConverter to elevate the correlation header into the JMS Message; you would then configure the aggregator to correlate on your custom header instead of the default.

        The custom MessageConverter would need to be wired into the channel's JmsTemplate; the namespace currently doesn't support that so you'd have to define the channel as a <bean/>.

        Of course, it would be much simpler to just have a unique backing queue for each node instance.


        • #5
          Gary thanks much for your prompt response.
          Regarding your comment:
          >> Of course, it would be much simpler to just have a unique backing queue for each node instance.
          Our splitter/aggregator logic will be sitting on JVMs (on appserver cluster) and we may need several aggregators per JVM craeted dynamically based on demand. I am not sure if I understood correctly, but in such situation we can't back up each aggregator with a JMS queue (which is statically configured). Is that right?


          • #6
            I assume you mean "can't back up each channel with a JMS queue".

            That is correct (if your broker requires static queue configuration); the other solution should work though, it just requires a little manual configuration (and a custom message converter based on the SimpleMessageConverter to add the header for the selector).