Announcement Announcement Module
Collapse
No announcement yet.
Issue with commit interval Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • Issue with commit interval

    Hello All,

    I am reading a huge file with 500 thousand records line by line and dumping it into a staging table. There are two type of records in the file. They are with prefix EMP and SH.
    Sample file could be as shown below:

    EMP|1....
    SH|1....
    SH|2....
    EMP|2...

    When the commit interval is 1 everything is working fine. It dumps all of those records into respective table.
    When i changed the commit interval to 4 things are getting messed up. The first read of the above sample file is returning me 4 objects but with EMP|2 twice and SH|2 twice. The EMP|1 and SH|1 seems to be overwritten. Hence i am missing some records in the table and am seeing duplicate records instead.

    The reader is reading all the records right. But when the control goes to the writer im seeing duplicates with missing records.
    Please let me know if im doing anything incorrect.

    SimpleChunkProvider doRead method is sending out the right bean with right data. But when it comes to ChunkOrientedTasklet code snippet
    Chunk<I> inputs = (Chunk<I>) chunkContext.getAttribute(INPUTS_KEY);
    if (inputs == null) {
    inputs = chunkProvider.provide(contribution);
    if (buffering) {
    chunkContext.setAttribute(INPUTS_KEY, inputs);
    }
    }
    when it pulls out inputs from chunkProvider it is having duplicates of same record label type with other records missing.
    Last edited by sushmapandith; Mar 12th, 2010, 11:40 AM. Reason: Adding more information
Working...
X