Announcement Announcement Module
Collapse
No announcement yet.
ErrorHandler strategy for 2.0 ? Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • ErrorHandler strategy for 2.0 ?

    Dear all,

    Do you think it might be possible to have JIRA INT-628 (ErrorHandler Strategy) implemented for the 2.0 release?

    My flows are quite long and thus I need to be able to control quite precisely how to handle errors at each endpoint of my flow. This JIRA regarding error handling at a low level (I mean, channel or endpoint) is definitely a major prerequisite of the push to Production of my project.

    Pierre

  • #2
    Could you please elaborate on your use case in more details. There are many ways to handle errors in SI, some always been there some new, so more details would help.

    Comment


    • #3
      Hi Oleg and thank you for your reply.

      Here's one of the many use cases where I'd need some error handling. I provided a sample of my XML configuration below (some channels are defined in other files so don't bother looking for them).

      There is a succession of endpoints, and for each endpoint (or channel), I would have liked to configure how to handle the errors.

      All my service activators may throw exceptions: depending on the service activator (and possibly, but not essentially, depending on the exception), I would like to send an email; for others, route the message to another channel or simply drop the message. Same thing for my customized splitters and transformers.
      All these behaviours could be quickly implemented if there was a configurable "error-channel" attribute for each of these endpoints.

      Note that I'm working in a synchronized flow here. But even in an unsynchronized flow, more customization would really be nice to handle errors differently depending on where and why the message failed to be processed.


      Any solution I would have missed to do this?


      Code:
      <int-file:inbound-channel-adapter channel="fileChannel" directory="/Tests" filename-pattern=".*\.xml" auto-startup="false">
      	<int:poller>
      		<int:interval-trigger interval="1000" />
      	</int:poller>
      </int-file:inbound-channel-adapter>
      
      <int:channel id="fileChannel" />
      	
      <int-file:file-to-string-transformer input-channel="fileChannel" output-channel="fileContentChannel" />
      
      <int:channel id="fileContentChannel" />
      
      <int:header-enricher input-channel="fileContentChannel" output-channel="headersChannel">
      	<int:header name="rawInputMessage" expression="payload" />
      	<int:header name="auditProcess" value="EPS" />
      </int:header-enricher>
      	
      <int:channel id="headersChannel" />
      	
      <int:header-value-router input-channel="headersChannel" header-name="technicalRedelivery" default-output-channel="noRedeliveryChannel">
      	<int:mapping value="true" channel="duplicateChannel" />
      </int:header-value-router>
      
      <int:channel id="noRedeliveryChannel" />
      
      <int:chain input-channel="noRedeliveryChannel" output-channel="endOfChainChannel">
      
      	<int:service-activator ref="trnGenerator" method="generateTrni" />
      
      	<int:service-activator ref="auditMessagePersistence" />
      
      	<int-xml:unmarshalling-transformer unmarshaller="JaxbMarshaller" />
      
      	<int:splitter method="splitMessage">
      		<bean class="com.RuleSolverFlowDetailsSplitter">
      			<property name="dataSource" value="DKG" />
      			<property name="ruleSolverTreeName" value="DKG" />
      		</bean>
      	</int:splitter>
      
      	<int:service-activator ref="trnGenerator" method="generateTrno" />
      
      </int:chain>
      	
      <int:channel id="endOfChainChannel">
      	<int:interceptors>
      		<int:wire-tap channel="auditPendingChannel"/>
      	</int:interceptors>
      </int:channel>
      
      <!-- To be completed... -->
      <int:bridge input-channel="endOfChainChannel" output-channel="nullChannel" />
      
      <!-- CHANNELS ASIDE -->
      	
      <int:channel id="duplicateChannel" />
      
      <int-xml:unmarshalling-transformer unmarshaller="JaxbMarshaller" input-channel="duplicateChannel" output-channel="auditCheckDuplicateChannel"/>
      
      <!-- BEANS DEFINITIONS -->
      
      <bean id="trnGenerator" parent="abstractTrnGeneratorServices">
      	<property name="sequenceName" value="EPS" />
      	<property name="prefix" value="EPS" />
      </bean>
      Thank you!

      Comment


      • #4
        Any idea how to handle errors at the endpoint level ?

        Comment


        • #5
          Well, the $errorChannel could always be specified with header-enricher,but you must understand how that works.
          If you are in the sync flow (all endpoints connected via direct channels), then the entire flow is on the same thread, so the exception will be re-thrown back to the original caller and you can do whatever it is you want to do with it including notifying others about what happened.
          If you are in the async flow where a thread does not know who the original caller is if it catches the exception it will be looking for the $errorChannel header.
          Does that help?

          Comment


          • #6
            Thank you Oleg for these details.
            I know how error handling currently works in sync and async flows, but the problem is that this behaviour doesn't allow enough customization:
            • In a sync flow, depending on the endpoint from which the exception is thrown, I would like the behaviours to be different.

              For instance, if the splitter fails with an exception, I would like to send an email with the message "Warning! The splitter failed for the reason: + exception.getMessage()". However, if one of the service-activator "trnGenerator" fails, I would like to call another service-activator. etc...

              If I catch the Exception at the inbound-channel-adapter point, I will only get a MessagingException and won't be able determine in which endpoint it failed (or I would have to look into channel history, which is definitely not an elegant solution) and decide the right action to do. Or if I eventually manage to do so, all the decision would be stored in one single place.

              If there was an error-handler attribute for each endpoint (service-activator, splitter, ...), this would be much more flexible, wouldn't it?

            • In an async flow, to have this flexibility, I would need to have an header-enricher to set the $errorChannel before each endpoint for which I need a specific behaviour.
              Again, an error-handler attribute for each endpoint would be much simpler.

            All in all, having the "error-handler" attribute for each endpoint would allow more flexibility and there wouldn't be any difference any more between sync and async flows regarding error handling.

            That's why I found JIRA INT-628 so interesting and was wondering if there was any plan to implement it.

            Note: Some default implementations of error handlers could also be suggested to the user such as a "moveToChannel" or "dropMessage" handlers.

            What do you think?
            Last edited by plecesne; Sep 2nd, 2010, 08:11 AM.

            Comment


            • #7
              Any comment?

              Comment


              • #8
                This would be a great feature. I've had to make my sync flows async just to get the error handling behaviour I want. With the addition of header enrichers for the required error channel

                Comment


                • #9
                  Would the new "exceptionMapper" support on a gateway potentially handle your use-case? (see: http://static.springsource.org/sprin...#gateway-proxy)

                  It has been our goal to use "gateways" (proxies) as request/reply components that hide the details of sync vs. async downstream message flows (e.g. if a error occurs in an async flow, the gateway will still receive it and then rethrow is so that it seems no different than if it happened synchronously). The 'exceptionMapper' then provides a hook to convert errors to valid Messages (but typically "error messages" or "faults") rather than having them re-thrown to the caller.

                  The problem with a general error-handler strategy is that it might be at odds with our use of the errorChannel to provide that consistency between sync and async flows. Also, different components can react differently. For example, you mentioned "drop mesages" as a strategy option, and that is exactly one of the options for a Filter, but it does not seem appropriate on other components (e.g. "Service Activator").

                  Finally, consider the fact that those various components can actually share the same "error-channel", but that channel can itself have a Router connected. This way, you can rely on the same messaging patterns for your error flows. Certain types of error might result in an email while other types are only logged, etc.

                  Do any of these ideas seem worth exploration? I hope it doesn't seem like we're just trying to avoid adding that one feature... it's actually a relatively trivial thing, but we are often walking a very thin line between flexibility and overwhelming people with too many options for handling a particular situation.

                  Thanks,
                  Mark

                  Comment


                  • #10
                    I'm happy with what I've done (I think) and defo understand trying to not have too many options, it's just that I've had to make my flow async in order to achieve what I want which I had no real reason otherwise to do.

                    In my use case I have a message flow using gateway proxy facade bean that ultimateley sends a web service request and on a soap fault I need to log the fault to the database as well as an exception being thrown to the caller. I suppose I could have used AOP for this which may or may not be better but what I currently have done is made my gateway's request channel a queue channel to break the sync so faults are put on the errorChannel. In fact I override the error channel header in these cases so the faults go to my own error channel. I then process those errors routing to 2 different channels. The first one logs the fault to the database and the second one gets the reply channel from the failed message and puts an ErrorMessage on that channel so the error is thrown to the caller as per how it would if everything was sync.

                    In summary I think there are use cases where a developer may need to do more than just have the error re-thrown and the hook to do this to me is either making use of the errorChannel or using AOP. I don't think exceptionMapper allows me to achieve this behaviour.
                    Last edited by rhart; Sep 7th, 2010, 06:32 AM.

                    Comment


                    • #11
                      Hi Mark and thank you for your detailed response.

                      I completely understand your point of view regarding all the feature requests you may receive, and especially your will to keep SI clean without useless options.

                      However, I haven't been convinced by any of the error handling solutions I've been suggested so far, either because they did not match my use case (error-channel working only in asynchronous flows) or because they are way too complicated for what error-handling should be.

                      Regarding what you told me:
                      1. We can't expect the developers to use gateways or switch their flows to asynchronous flows only because they need to add some error handling to their flows. Error handling is something that can be implemented at the end of a project, just to add some more reliability to the flow.
                        What would be the cost to refactor everything to use gateways?

                        In my case, I have a synchronous flow of service-activators, splitters, transformers, ... and it doesn't seem so simple to change all my endpoints to gateways (or add gateways everywhere between them) only to be able to handle exceptions correctly.

                        Moreover --but this may be due to my limited knowledge on the subject--, I don't see how I can use gateways as the beginning of my flow is not Java code but a channel adapter. My Java code is actually only limited to a few POJO splitters and transformers. The rest is only SI configuration.

                      2. The exceptionMapper returns a "valid" message, so the flow keeps going on its normal flow, which can sometimes be what one wants to do. On the other hand, this solution doesn't allow to get out of the normal flow to do something different (filter, route, log, ...)

                      3. The "error-channel" solution only works in an asynchronous flow if I understood correctly. And I definitely can't work asynchronously here as my flow is executed in a transactional context.
                        Moreover, adding a header-enricher to set the $errorChannel before each necessary endpoint is only limited to simple cases: we couldn't have only certain exceptions caught and sent to the errorChannel and others thrown back to the callers for example.
                        Same for the router connected to the errorChannel: this would certainly allow to do different actions depending on the type of the exception, but not depending on from which endpoint it failed.

                      4. Indeed, a Service Activator shouldn't have the role of a filter. But this would not be the case here: the Service Activator would keep its role, but when an exception occurs, then the error handler could possibly be a filter, a router, or something else.
                        Maximum flexibility to handle almost all use cases, simple implementation, respecting messaging patterns, and intuitive enough for all users to use quickly and easily. For an essential feature, this doesn't seem such a bad solution

                      Although I don't really understand why this would be at odds with the current use of the errorChannel, I'm willing to accept that the error-handler attribute on endpoints may not be the best solution to handle exceptions -- I trust your experience -- but then SI should offer some other simple, intuitive and flexible solution which works as well in synchronous and asynchonous flows without having to change much of the existing spring configuration (and java code, of course).

                      I've been astonished so far by the simplicity to implement a new flow in Spring Integration.
                      This would really too bad if adding "precise" error handling (a basic and essential feature in integration systems) causes users to break/refactor all their spring configuration or wonder for hours how to turn their code into something SI-compliant. This should be, in my opinion, as simple and intuitive as adding a wire-tap on a channel.


                      I hope I didn't offense anyone, this is really not my goal. I'm a big fan of Spring Integration and I really appreciate all the time you spend on this forum, helping and informing us.

                      Thanks again

                      Pierre

                      Comment


                      • #12
                        Hi Guys,

                        I am using the message driven adaptor with config below, to process messages as they arrive. The config essentially means
                        that there are 5 threads who are going 5 messages and each will follow their path of execution to do some processing, going
                        through transformation etc and in the end a service activator which does some business processing and place the response on
                        Topic using <jms:outbound-channel-adapter>

                        <jms:message-driven-channel-adapter
                        id="jmsRequestIn" destination="requestQueue" channel="router" max-concurrent-consumers="5"
                        acknowledge="transacted" connection-factory="connectionFactory" extract-payload="true">

                        <channel id="router">
                        <channel id="transformer">


                        Doubts:
                        1.I understand that any exception that happen in processing pipe( of channels ) will throw exception back to
                        jms:message-driven-channel-adapter as each execution path is single threaded.

                        2. How I handle the exception here as I do not have the anything like exception-mapper for jms:message-driven-channel-adapter,
                        which is available for gateway.

                        3. I want to handle the exception and then depending on the exception , I would like ideally like to retry or commit transaction with a
                        failure message on Topic. I am not able to achieve this unless I get some pointer point 2.

                        Any help will be of great help. And, ideally I would have loved what Pierre has raised for. But for the time being I will be fine with a
                        so;ution for 2.


                        Cheers

                        Comment


                        • #13
                          When the exception is throw back to jms:message-driven-channel-adapter I believe that depending on your JMS setup the transaction will be rolled back, the JMS message put back on the queue and will be retried. After the retry limit (that's configured JMS wise) is reached the message will then be put on the dead letter queue.

                          If you want to do something with the exception yourself and do different things on different exceptions you'd either have to do something with AOP or make the router channel a queue channel. If you do that then any exception that happens from there on in will be put on the error channel and you can do what you want. The thing to be aware of with that approach though is that as soon as the message hits the router channel the first transaction will be committed and a new thread will begin. Any retry logic can be handled by your SI config (e.g you might redirect it back to the router channel, or wait a bit and then reroute) but the message will never be rolled back onto the JMS queue.

                          Hope that helps and I'm making sense. I'm sure someone will correct me if I'm wrong

                          Comment


                          • #14
                            Originally posted by rhart View Post
                            When the exception is throw back to jms:message-driven-channel-adapter I believe that depending on your JMS setup the transaction will be rolled back, the JMS message put back on the queue and will be retried. After the retry limit (that's configured JMS wise) is reached the message will then be put on the dead letter queue.
                            Yes, that(above) will/should be the behaviour.

                            Originally posted by rhart View Post
                            If you want to do something with the exception yourself and do different things on different exceptions you'd either have to do something with AOP or make the router channel a queue channel. If you do that then any exception that happens from there on in will be put on the error channel and you can do what you want. The thing to be aware of with that approach though is that as soon as the message hits the router channel the first transaction will be committed and a new thread will begin. Any retry logic can be handled by your SI config (e.g you might redirect it back to the router channel, or wait a bit and then reroute) but the message will never be rolled back onto the JMS queue.

                            Hope that helps and I'm making sense. I'm sure someone will correct me if I'm wrong
                            Yeah, the kind of one I thought of but felt like not the ideal way. Ok, here are few things with the sample config

                            <jms:message-driven-channel-adapter
                            id="jmsRequestIn" destination="requestQueue" channel="router" max-concurrent-consumers="5"
                            acknowledge="transacted" connection-factory="connectionFactory" extract-payload="true">

                            <channel id="routerChannel">
                            <channel id="route1">
                            <channel id="route2">
                            <channel id="route3">
                            <channel id="outputChannel">

                            <channel id="transformer">

                            <!-- This will route the message to route 1,2 ,3 etc.. -->
                            <router ref="requestRouter" input-channel="routerChannel" default-output-channel="routerErrorChannel" method="route" />

                            <si:channel id="route1" >
                            <si:queue capacity="10"/>

                            </si:channel> and.. so on with a service activator at the end of chain.
                            1. I agree that the first transaction here would be committed as soon as message is put on the channels 1, 2 and so on. I will still live with it.
                            1. I tried adding number of threads to route1 channel using dispatcher element but I think they cant co-exist( i get error in config). I want to have something like say 4 threads processing messages on a given route.

                              And if I do not specify capacity then that would mean MD-Adapator threads will keep on putting the message on route(1,2,3). This is not nice if the server crashes and I will no messages left to reprocess.

                              Another thing is- suppose MD-Adaptor threads are getting 10 messages per sec and business process threads on 'route1/2/3' can process only 2 messages per second. Will MD-adaptor threads will block and wait for queue to have space for new messages or will it start mis-behaving. I will test this but if anyone know on top of the head.

                              On reroute/retry- Will there be contention among MD-Adaptor thread and thread on error channel , when they try to put the message on route if there is a one empty space.

                            Apologies for long message. Just want to make sure that I am thinking in right direction.

                            Comment


                            • #15
                              Hi Rhart/Oleg/Mark,

                              Can you please comment on doubts/concerns that I have posted? And please let me know if i am on right path.

                              Thanks a lot.

                              Comment

                              Working...
                              X