Announcement Announcement Module
Collapse
No announcement yet.
Eliminating duplicate records before writing to DB Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • Eliminating duplicate records before writing to DB

    Any suggestions on how to eliminate duplicate records before passing it to the writer.

    I tried implementing ItemWriter but its called as many times as the number of source records if injected into the chunk as writer.

    Thanks
    AN

  • #2
    I believe the only way to do is to dump the whole file into a temp table in step one. And then do a self join and a write to final database in second step.

    step 1: File to DB
    step 2: DB to DB

    Nik

    Comment

    Working...
    X