Announcement Announcement Module
Collapse
No announcement yet.
Step VS Tasklet VS Chunk Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • Step VS Tasklet VS Chunk

    I have a problem choosing between steps, tasklets, and chunks - which approach is best?

    I'll try and describe the Use Case in words, but if you want, I can send a simple test case which will describe the problem.

    Problem:
    I need to do a channelised read/process/write (a 3-step process), every few seconds.
    A list of channels could in all instances be injected.

    step 1 - "read":
    From database, channelised, using org.springframework.jdbc.datasource.lookup.Abstrac tRoutingDataSource.
    For each channel, read from the channelised database some ItemA, put these ItemAs in a list, and apply some Comparator which leaves me with one final ItemA. This, I return. This ItemA is also aware of which channel it was read from.

    step 2 - "process":
    For each channel, do a join with ItemA and some other ItemB in the channelised database (again, using AbstractRoutingDataSource) and assemble a DTO (ItemA.id joined on ItemB.itemAID) per channel. Some databases won't have an ItemB, so:
    number of DTOs <= number of channels.
    Return a list of DTOs where each DTO is channel-aware (looking at ItemB's channel this time).

    step 3 - "write":
    Store each DTO in a channelised cache. Some channels may not have a DTO so the channelised cache retains the previous entry.

    Dilemma:
    I'm a bit unsure on how to proceed. I have options: steps VS tasklets VS chunks, but which approach is the best?

    [1] Should "read", "process" and "write" be implementations of Item*Adapter, with the list of channels injected?
    E.g.
    Code:
    class ItemAReader extends ItemReaderAdapter<ItemA>
    // service injected as per super class requirements
    // channels injected, etc
    
            @Override
    	public ItemA read() throws Exception {
    		for (Channel channel : supportedChannels.getChannels()) {
    			ChannelContextHolder.setChannel(channel);
    			someList.add(invokeDelegateMethod());
    		}
    		// apply the Comparator on someList
    		
    		return someList.get(0);
    	}
    [2] Or instead of this chunked approach do I make each step a Step, where I save ItemA in a StepExecution context?

    Other things to consider that may point me in the right direction:
    • ItemA shouldn't be re-processed in a subsequent job, so subsequent jobs should probably know about the previous successfully-processed ItemA (in other words, if a DTO with ItemA.id=9 is already cached, do not cache it again. So, step1 may need to peek into the cache?)
    • If no ItemA can be found, end on step 1.
    • If no ItemB could be found in any of the channels, nothing would end up being cached, so end on step 2.
    • Step 3: there may be an issue with the cache, so just fail gracefully.
    • all 3 steps will happen in succession every few seconds (scheduled by Quartz)
    • this is not a mission critical system, so if a job fails for a certain ItemA (say, with id=3), the subsequent job needn't be retried for ItemA.id=3. The valid ItemA on subsequent run may be ItemA.id=8 (e.g. the "latest" ItemA)

    Thanks,
    Juan
    Last edited by opyate; Feb 23rd, 2010, 06:14 AM.

  • #2
    I think I should use "Steps with Tasklets" instead "Steps with chunks", for the following reason:

    step 1 should return an ItemA which is aware of the channel it was read from, hence ChannelisedItemADTO.
    But this forces the delegate to also return ChannelisedItemADTO, which it doesn't.

    Which leads me to a secondary question: Batch developers please take note: Is it worth me logging an enhancement request in JIRA so we can specify different parameterised types for ItemReader.read() and invokeDelegateMethod()? (We suspect you'll ignore us ;-) and tell us to refactor our service method to return said DTO alongside the POJO.)

    I'll use Tasklets (implementing StepExecutionListener) and set the channel in the context using the tip provided here:
    http://static.springsource.org/sprin...aToFutureSteps
    Last edited by opyate; Feb 23rd, 2010, 07:47 AM.

    Comment

    Working...
    X