Announcement Announcement Module
Collapse
No announcement yet.
Remote chunking with partial failure recovery Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • Remote chunking with partial failure recovery

    Dear all,

    I wonder whether remote chunking supports partial failure recovery. For example, assume one job has 100 items and some items are not successfully handled by slaves (maybe slaves are down). Can Master or middleware (e.g. ActiveMQ) detect the failure and only redeliver the failed items?

    My configuration of remote chunking follows this tutorial: http://www.opencredo.com/blog/spring...emote-chunking. I am using Spring 2.0.0 and ActiveMQ 5.8.0

    Thank you.

  • #2
    Remote chunking requires a middleware that provides guaranteed delivery for this very reason. If a slave is down, it becomes the middleware's responsibility to maintain that message until it can be handled by another slave.

    Comment


    • #3
      Originally posted by mminella View Post
      Remote chunking requires a middleware that provides guaranteed delivery for this very reason. If a slave is down, it becomes the middleware's responsibility to maintain that message until it can be handled by another slave.
      I have the same problem and I don't understand exactly what you mean with the middleware's responsibility.
      I didn't find any solution or documentation regarding the transaction management on slaves. Can I add batch rollback policies(retry, skip) on slaves?

      Comment


      • #4
        Just some more infos regarding my implementation.
        Like I said I have a remote chunking implementation. The Master read some chunks of data and send them via RabbitMq to be processed by the slaves. Each slave take one chunk to process it. My question is, what happens when only one item from the entire chunk fails? All the transaction is rollbacked and a reply message is sent with a fail status? Is there a possibility to skip only that failed item and process the other items from the chunk? (something like skip-policy).
        Thank you

        Comment

        Working...
        X