Announcement Announcement Module
No announcement yet.
Newbie - Advice needed for POC Page Title Module
Move Remove Collapse
Conversation Detail Module
  • Filter
  • Time
  • Show
Clear All
new posts

  • Newbie - Advice needed for POC


    I am a batch Newbie looking to do a POC to replace a legacy job using Spring Batch. I have been doing some learning but I would like some advice on the functional side.

    I have a table which has around 1 million MasterRecords in it and each one of these MasterRecords has 1 record each in 5 views (lets call them SlaveRecords1, SlaveRecords2 ... SlaveRecords5). The cost of retrieving a SlaveRecord using a MasterRecordId is low although these SlaveRecords views are really humongous because they are built on other views/tables and PL/SQL functions. The requirement is simple - read all master records and corresponding slave records and write them to a file.

    An initial idea I have is to slice the MasterRecords table into partitions and use PartitionedStep. But with in each Step, how do I efficiently retrieve the SlaveRecords?. The way I currently perceive it, after retrieving each MasterRecord I have to make 5 queries to retrieve the 5 corresponding slave records. Are there any patterns in Spring batch to do this in a better way? I have been reading about driving query pattern but even in that case I still have to make 5 queries each for every record ( A join is really hard bcos of the complexity of the views)

    Any help is greatly appreciated.


  • #2

    After partitioning the MasterRecords, to retrieve slave records you can make use of StoredProcedure.