Announcement Announcement Module
Collapse
No announcement yet.
SI Multithreading has really weird behaviour Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • SI Multithreading has really weird behaviour

    I have the following setup:
    Code:
    <jms:channel id="inboundStringChannel" queue="rowsQueue" message-driven="false" />
    <jms:channel id="inboundSaleDataChannel" queue="dataQueue" message-driven="false" />
    
    	<beans:bean id="stringToSaleDataThreadPoolTaskExecutor" class="org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor">
    		<beans:property name="corePoolSize" value="15" />
    		<beans:property name="maxPoolSize" value="20" />
    		<beans:property name="waitForTasksToCompleteOnShutdown" value="true" />
    	</beans:bean>
    
    <splitter id="fileToStringsSplitter" input-channel="inboundFileChannel" output-channel="inboundStringChannel" ref="inboundFileToStringsSplitter" />
    
    	<transformer id="stringToSaleDataTransformer" input-channel="inboundStringChannel" output-channel="inboundSaleDataChannel" ref="inboundStringToSaleDataTransformer" >
    		<poller task-executor="stringToSaleDataThreadPoolTaskExecutor" time-unit="SECONDS" fixed-delay="30" max-messages-per-poll="15">
    			<transactional transaction-manager="txManager" />
    		</poller>
    	</transformer>
    To test multithread, i have in my transformer something similar to:
    Code:
    long threadId = Thread.currentThread().getId();
    LOG.info("Thread #" + threadId + " started...");
    
    try {
    			Thread.sleep(1000000000);
    		} catch (InterruptedException e) {
    		}
    What I expect from the config above:
    15 concurrent transformers processing data from JMS (there is like 2000+ data available for processing).
    I'm not sure, if SI can distribute only CPU bound threads or virtual as well, but in case if those are CPU bound - I expect at least 4 threads running (got 4 CPUs).

    What I get while testing:
    SI runs that many threads, as it wants, but not more then 3.
    1st run - I got 3 simulatenous threads
    2nd run - I got 1 threads
    3rd run - I got 2
    4th run - I got again 1

    Questions & unclear moments:
    1) Are threads which allocates SI CPU bound or not ?
    2) Why SI runs random amount of threads each time ?
    3) How to get the effect i expect ?

  • #2
    No; the threads are not CPU bound.

    The poller is only scheduling one thread every 30 seconds; it will take 7.5 minutes to get to 15 threads; and then only if all the threads are busy at the time a new one is scheduled. The core pool size is not an initial size it's a size the pool won't shrink below.

    For a use case like this, it's better to use a message-driven channel and control the concurrency with the listener container.

    Or, decrease the poller delay to a few hundred ms.

    Comment


    • #3
      Thanks a lot Gary !

      I see, misunderstood the way it works before.
      Probably because of "max-messages-per-poll". It made me confused, since i thought that will be the main factor how many threads to allocate based on amount of messages received.

      Comment


      • #4
        No; max-messages-per poll means when a scheduled thread polls the channel it will try to fetch that number of messages before going back to sleep.

        Comment

        Working...
        X