Announcement Announcement Module
Collapse
No announcement yet.
inconsistent message order when tcp-inbound using NIO Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • inconsistent message order when tcp-inbound using NIO

    I'm receiving single messages though a tcp-inbound-channel-adapter. The messages are multi-line, split by \r\n. The ByteArrayCrLfSerializer splits each line into a seperate message and they are sent onto a directChannel.

    However, each one seems to be sent by a different thread; depending on thread scheduling, they arrive out of order, even when delivering to a directChannel.
    • This only happens when using-nio="true" on the server tcp-connection-factory.
    • Setting the pool-size on the server factory to less than 3 results in a lock up, so I can't force it to use a single thread for servicing the split messages.
    • only one multi-line message is being recieved, so it's not packet arrival times (TCP ensures order anyhow)

    If you had a regular <splitter> connected to a directChannel, the split messages would always arrive in sequence. A Resequencer is only really for compensating for external services replying at different speeds, not for correcting internal race conditions.

    Input (single message): 1\r\n-2\r\n--3\r\n---4\r\n----5\r\n-----6\r\n------7\r\n-------8\r\n--------9

    expected output, one line per message:
    1
    -2
    --3
    ---4
    ----5
    -----6
    ------7
    -------8
    --------9

    actual output (threadding dependent):
    -2
    --3
    ---4
    ----5
    -----6
    -------8
    --------9
    1
    ------7
    Last edited by pauladamson; Dec 16th, 2010, 12:24 PM. Reason: added (solved) to title

  • #2
    The use of NIO in the channel adapters/gateway is specifically to avoid dedicating a thread to each socket. You are correct in that this introduces the indeterminate behavior you are seeing. However, NIO is only needed when supporting a very large number of connections.

    We recognized this situation and considered adding strict sequencing within the adapters; however, this would have to be conditional because there are use cases that explicitly do not need it, and actually require each message to be processed as soon as possible. Adding this conditional sequencing logic would have added complexity,

    Instead, we added a sequence header so that a resequencer could be added downstream, if needed.

    Unless you need to support a very large number of concurrent connections, set using-nio=false and use an async handoff.

    HTH.

    Comment


    • #3
      Hi Gary.

      Seeing different behaviour between the implementations made me wonder if it was a defect.
      Thanks for the great insight into the though behind this!

      Sorted.

      Comment


      • #4
        Great!

        We will clarify this behavior in the documentation.

        https://jira.springsource.org/browse/INT-1692

        Comment

        Working...
        X