Announcement Announcement Module
Collapse
No announcement yet.
Chunk parallelization using org.springframework.core.task.SimpleAsyncTaskExecu tor Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • Chunk parallelization using org.springframework.core.task.SimpleAsyncTaskExecu tor

    Hi,
    We were trying to implement multi-threading at step level i.e. in chunk. But we are not able to achieve that.
    Below is snippet from our job

    < <step id="wsStudentGenStep">
    <tasklet task-executor="taskExecutor">
    <chunk reader="wsStudentItemReader" processor="wsReqPrcsr"
    writer="outputWriter" commit-interval="4" skip-limit="20">
    <skippable-exception-classes>
    <include class="java.lang.Exception" />
    </skippable-exception-classes>
    </chunk>
    <listeners>
    <listener ref="WSListnr" />
    </listeners>
    </tasklet>
    </step>
    <job id="wsStudent">
    <step id="wsStudentFileGenIntialStep" parent="initiateStep"
    next="wsStudentFileGenStep" />
    <step id="wsStudentFileGenStep" parent="wsStudentGenStep" />
    </job>
    <beans:bean id="taskExecutor"
    class="org.springframework.core.task.SimpleAsyncTa skExecutor">
    <beansroperty name="concurrencyLimit" value="2"/>
    </beans:bean>
    The reader object is org.springframework.batch.item.database.JdbcCursor ItemReader.
    Please guide if we are missing any thing in configurations.

    We are getting error with above configuration

    org.springframework.jdbc.UncategorizedSQLException : Attempt to process next row failed; uncategorized SQLException for SQL []; SQL state [null]; error code [-4470]; [jcc][t4][10120][10898][3.59.81] Invalid operation: result set is closed. ERRORCODE=-4470, SQLSTATE=null; nested exception is com.ibm.db2.jcc.am.SqlException: [jcc][t4][10120][10898][3.59.81] Invalid operation: result set is closed. ERRORCODE=-4470, SQLSTATE=null
    Last edited by ash-bang; May 8th, 2013, 08:20 AM.

  • #2
    We found a solution to the above mentioned problem. Instead of JdbcCursorItemReader we had to use JdbcPagingItemReader. And we were able to run the parallelization.

    But again we got a problem. Our query involves two tables and the primary key is composite key. So we are not able to get what to put on sortkey. And also in org.springframework.batch.item.database.support.Ab stractSqlPagingQueryProvider class; the below method removes the alias, so it is giving BAD SQL GRAMMAR Exception as that sortkey is present in both tables.
    public String getSortKeyWithoutAlias()
    {
    String sortKey = getSortKey();
    int separator = sortKey.indexOf(46);
    if (separator > 0) {
    int columnIndex = separator + 1;
    if (columnIndex < sortKey.length())
    sortKey = sortKey.substring(columnIndex);
    }

    return sortKey;
    }

    Comment

    Working...
    X