Announcement Announcement Module
Collapse
No announcement yet.
Error handling in remote chunking scaling strategy with spring integration Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • Error handling in remote chunking scaling strategy with spring integration

    Hi,

    I am new to this forum and I am looking for the solution to my problem, any suggestion will be very appreciated.
    I have an application that has to process huge amount of data. The remote chunking scaling strategy of spring batch and spring integration(the master and slaves topology) is used.
    The master is a spring job that reads data from the DB and is sending chunk requests into the request channel that is integrated with request JMS queue. From other side, there is a service activator(part of slave process) linked to the request channel and is processing&writing the chunks from the queue and outputs the chunk response into chunk reply channel linked to another JMS queue.

    Below is an configuration of the slave:

    Code:
    <bean id="connectionFactory" class="org.apache.activemq.ActiveMQConnectionFactory" >
      <property name="brokerURL">
           <value>tcp://localhost:61616</value>
        </property>
    </bean>
      
    <int-jms:inbound-channel-adapter id="jmsRequests" connection-factory="connectionFactory"
             destination-name="requests" channel="requests" >
        <integration:poller fixed-delay="500"></integration:poller>
    </int-jms:inbound-channel-adapter>
     
    <integration:channel id="requests"/>
      
    <integration:service-activator id="srvActivator" input-channel="requests"
                 output-channel="replies" ref="chunkHandler" method="handleChunk"/>
    <integration:channel id="replies"/>
       
    <int-jms:outbound-channel-adapter id="outgoingReplies" connection-factory="connectionFactory"
             destination-name="replies" channel="replies">
    </int-jms:outbound-channel-adapter>
    Everything works fine and the data is being processed.
    Now the problem I have is that if an item from the chunk fails during processing, the whole chunk is rolledbacked..
    What I want to achieve is to handle this failure/exception as a retry or skip mechanism so the chunk being processed to not be affected.

    What I found so far is that the int-jms:inbound-channel-adapter has error-channel and error-handler tags that could be used to solve my problem...

    Can you please suggest how can I achieve the handling of error/exception of an item from a chunk and not rollback the whole chunk being processed? Also I would like to reprocess the failed items at the end or in a different manner according to your suggestions.

    Thanks a lot!
    Regards,
    Andrei

  • #2
    Take a look at this section of the reference manual.

    You can add an advice to your service activator - either a retry advice - with a recovery strategy that skips the chunk after some number of retries (the default recoverer logs a WARN, but you can use a custom recoverer to do more.

    If you don't want the complexity of retry you can use an expression evaluating advice with an onFailure expression.

    There are examples of using these advices in the retry-and-more sample.

    Comment


    • #3
      Thanks a lot for your reply. I will try your suggestions and get back with the results.
      Once again, thank you!

      Comment


      • #4
        Suggestion of Gary Russell is useful and I personally make sure it. Its really worthy and informative with useful suggestions. Its authentic solution for chunking scaling strategy.

        Bench Scales
        Last edited by Adan Bailey; Aug 30th, 2013, 01:08 AM.

        Comment

        Working...
        X