Announcement Announcement Module
Collapse
No announcement yet.
Multi Thread issue? Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • Multi Thread issue?

    Hi,

    Appreciate if someone can help solve the following issue. Basically, the application reads a CSV file and passes a group of lines at a time downstream for processing (e.g. DB insert)

    My config:
    Code:
    	<file:inbound-channel-adapter directory="${dir.sdr}" 
    		auto-create-directory="true"
    		id="sdrFilesIn"
    		filename-regex=".*SDR">
    		<si:poller fixed-delay="1000" max-messages-per-poll="10" task-executor="filePolers"/>
    	</file:inbound-channel-adapter>	
    	<task:executor id="filePolers" pool-size="2"/>
    	
    	<file:outbound-gateway directory="file:${dir.processed}" request-channel="sdrFilesIn"
    		auto-create-directory="true" delete-source-files="true"
    		reply-channel="sdrFile">
    	</file:outbound-gateway>
    	
    	<si:service-activator input-channel="sdrFile" 
    		ref="csvToSdr" method="toSdrs">
    	</si:service-activator>
    	<bean id="csvToSdr" class="com.syniverse.xdr.CsvToSdr">
    		<property name="batchSize" value="100"/>
    		<property name="outChannel" value="sdrChannel"/>
    	</bean>
    	
    	<si:channel id="sdrChannel"/>
    	
    	<si:router input-channel="sdrChannel" ref="sdrRouter" method="route" />
    	<bean id="sdrRouter" class="com.syniverse.xdr.SdrRouter"/>
    	
    	<si:channel id="acctChannel">
     	    <si:dispatcher task-executor="dbWorkers"/> 
    	</si:channel>
    	<si:channel id="alertChannel">
                <si:dispatcher task-executor="dbWorkers"/> 
    	</si:channel>
    	<task:executor id="dbWorkers" pool-size="2"/>
    
    	<si:service-activator input-channel="acctChannel" ref="acctMockInsert" method="batchInsert"/>
    	<bean id="acctMockInsert" class="com.syniverse.xdr.test.SdrMockInsert"/>
    	
    	<si:service-activator input-channel="alertChannel" ref="acctMockInsert" method="batchInsert">
    	<bean id="alertMockInsert" class="com.syniverse.xdr.test.SdrMockInsert"/>
    Everything is working as expected until I added items in red. It seems the SdrMockInsert stops receiving more items right after the file parsing is complete. Below is the sample outputs of working and non-working cases:

    Working
    Code:
    2011-08-24 14:45:11,421 [filePolers-2] INFO : xdr.CsvToSdr:32 - Parsing batches of lines from file: C:\workspace\SdrLoader\processed\one.SDR
    2011-08-24 14:45:11,453 [filePolers-2] INFO : test.SdrMockInsert:18 - Inserted 100 items. Total: 100
    2011-08-24 14:45:11,468 [filePolers-2] INFO : test.SdrMockInsert:18 - Inserted 100 items. Total: 200
    2011-08-24 14:45:11,484 [filePolers-2] INFO : test.SdrMockInsert:18 - Inserted 100 items. Total: 300
    ...
    2011-08-24 14:45:12,531 [filePolers-2] INFO : test.SdrMockInsert:18 - Inserted 100 items. Total: 11100
    2011-08-24 14:45:12,531 [filePolers-2] INFO : test.SdrMockInsert:18 - Inserted 61 items. Total: 11161
    2011-08-24 14:45:12,531 [filePolers-2] INFO : xdr.CsvToSdr:56 - 11161 SDRs read
    Non-working:
    Code:
    2011-08-24 14:57:47,937 [filePolers-2] INFO : xdr.CsvToSdr:32 - Parsing batches of lines from file: C:\workspace\SdrLoader\processed\one.SDR
    2011-08-24 14:57:48,062 [dbWorkers-1] INFO : test.SdrMockInsert:18 - Inserted 100 items. Total: 100
    2011-08-24 14:57:48,062 [dbWorkers-1] INFO : test.SdrMockInsert:18 - Inserted 100 items. Total: 200
    2011-08-24 14:57:48,078 [dbWorkers-2] INFO : test.SdrMockInsert:18 - Inserted 100 items. Total: 300
    ...
    2011-08-24 14:57:48,515 [dbWorkers-2] INFO : test.SdrMockInsert:18 - Inserted 12 items. Total: 3838
    2011-08-24 14:57:48,843 [filePolers-2] INFO : xdr.CsvToSdr:56 - 11161 SDRs read
    2011-08-24 14:57:48,515 [dbWorkers-1] INFO : test.SdrMockInsert:18 - Inserted 12 items. Total: 3850
    2011-08-24 14:57:48,906 [dbWorkers-1] INFO : test.SdrMockInsert:18 - Inserted 0 items. Total: 3850
    2011-08-24 14:57:48,906 [dbWorkers-1] INFO : test.SdrMockInsert:18 - Inserted 0 items. Total: 3850
    ...
Working...
X