Announcement Announcement Module
Collapse
No announcement yet.
SourcePollingChannelAdapter trigger rate problem Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • SourcePollingChannelAdapter trigger rate problem

    Hi there,

    I've manually (programatically) set up a SourcePollingChannelAdapter coupled to a file-scanning message source that has a trigger set to poll every x minutes. This works nicely, apart from the fact that the polling rate is ignored if the request and response from the endpoint gateway succeeds. So, for example, I send a message of filtered files to a web-service via an outbound HTTP entity (HttpRequestExecutingMessageHandler). If the web service is down, the trigger fires at the predetermined rate. If the web service is up (the files get processed, and a void or 200 response is sent), the trigger immediately fires again, despite the fixed-rate set to x minutes.

    I'm using the DynamicTrigger code from this site to make sure I can set the rate at runtime, even though I technically don't - the rate is fixed at trigger construction: https://gist.github.com/1720091

    So I add debug statements into the trigger's nextExecutionTime() method to see when this method is called and whether the trigger is correctly processed. So, with a "down" web service:

    Code:
     INFO [main] - lastScheduledExecutionTime::null
     INFO [main] - nextExecutionTime::Wed Mar 07 12:57:23 GMT 2012
     INFO [task-scheduler-1] - 121 eligible files from /datapath
     INFO [task-scheduler-1] - Processing 121 directories...
     INFO [task-scheduler-1] - ... done.
    ERROR [task-scheduler-1] - org.springframework.integration.MessageHandlingException: HTTP request execution failed for URI [http://localhost/consumer/]
     INFO [task-scheduler-1] - lastScheduledExecutionTime::Wed Mar 07 12:57:23 GMT 2012
     INFO [task-scheduler-1] - nextExecutionTime::Wed Mar 07 12:58:33 GMT 2012
    All good. And with an "up" web service:

    Code:
     INFO [main] - lastScheduledExecutionTime::null
     INFO [main] - nextExecutionTime::Wed Mar 07 13:02:56 GMT 2012
     INFO [task-scheduler-1] - 121 eligible files from /datapath
     INFO [task-scheduler-1] - Processing 121 directories...
     INFO [task-scheduler-1] - ... done.
     INFO [task-scheduler-1] - 121 eligible files from /datapath
     INFO [task-scheduler-1] - Processing 121 directories...
     INFO [task-scheduler-1] - ... done.
     INFO [task-scheduler-1] - 121 eligible files from /datapath
     INFO [task-scheduler-1] - Processing 121 directories...
     INFO [task-scheduler-1] - ... done.
     INFO [task-scheduler-1] - 121 eligible files from /datapath
     INFO [task-scheduler-1] - Processing 121 directories...
     INFO [task-scheduler-1] - ... done.
    Each of these polls is kicked off immediately following a successful processing stage, and not after the fixed trigger rate.

    Any thoughts? I'm happy to supply lots more info if required.

  • #2
    Try setting 'max-messages-per-poll' to 1 (or however many executions you want *within* each triggered poll). The idea is like this:

    Code:
    poll() {
      for (i..maxMessagesPerPoll) {
        msg = receive()
        if (msg == null) break
        send(msg)
      }
    }
    So, the trigger determines when each poll() occurs, but within poll, you can receive multiple messages. The idea is that you might want a quiet time of 10 minutes between polls of the file system, but if 1,000 files arrive during that quiet time, you want to grab them all and process (rather than waiting 1,000 * 10 minutes to process the whole group).

    Hope that helps.
    -Mark

    Comment


    • #3
      Have you set max-messages-per-poll to > 1?

      (An exception stops the poll but a good result will continue to call the message source either until mmpp is reached, or the message source returns null).

      Comment


      • #4
        Hi guys,

        Thanks for the quick replies. I tried setting maxMessagesPerPoll to 1, but this made no difference. I think this is because I'm using a custom MessageSource that grabs a set of files in once go, and constructs a single message from this collection, so that property wouldn't really work here.

        It's strange that the trigger only happens correctly when no response is received. This sentence interests me though: "or the message source returns null". Why would the trigger continue to be polled continually (i.e. not at the set fixed rate) when a response is successful?

        Comment


        • #5
          Because of the reasons that Mark stated above; however, if mmpp == 1, that should terminate the loop rather than the null result (see Mark's pseudo code above).

          Can you set a breakpoint in AbstractPollingEndpoint.Poller.run(), and check the mmpp is set properly - maybe step through and see why your MessageSource is being called again after success?

          Comment


          • #6
            I stepped through the polling code, but couldn't see anything that was immediately obvious.

            My main thought is why the trigger's nextExecutionTime() method isn't being called when the send/recieve is successful. What do you think the significance of this is?

            Comment

            Working...
            X