Announcement Announcement Module
Collapse
No announcement yet.
how to avoid reading the same record in Parallel Processing Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • how to avoid reading the same record in Parallel Processing

    Hi,

    I am trying to implement batch parallel processing for below requirement. But all the threads are reading all the records. I need to implement like each thread should read only one record.

    Reading the customer information from database, processing and updating the status in the DB. All the stuff should be implement in parallelly. I am using JpaPagingItemReader.

    <job id="ncyJob" >
    <split id="ncyJobSplit" task-executor="taskExecutor">
    <flow>
    <step id="ncyJobSplitOne" >
    <tasklet allow-start-if-complete="false" >
    <chunk reader="ncyReader" processor ="ncyProcessor" writer="ncyWriter" commit-interval="100" />
    </tasklet>
    </step>
    </flow>
    <flow>
    <step id="ncyJobSplitTwo" >
    <tasklet allow-start-if-complete="false" >
    <chunk reader="ncyReader" processor ="ncyProcessor" writer="ncyWriter" commit-interval="100" />
    </tasklet>
    </step>
    </flow>
    <flow>
    <step id="ncyJobSplitThree" >
    <tasklet allow-start-if-complete="false" >
    <chunk reader="ncyReader" processor ="ncyProcessor" writer="ncyWriter" commit-interval="100" />
    </tasklet>
    </step>
    </flow>
    </split>
    </job>

  • #2
    If your ncyReader is thread safe you can use a task executor in the step (instead of a split). Otherwise you need a partitioning approach. All is discussed in the user guide in the section on scalability.

    Comment

    Working...
    X