Announcement Announcement Module
No announcement yet.
reading from multiple files of different format Page Title Module
Move Remove Collapse
Conversation Detail Module
  • Filter
  • Time
  • Show
Clear All
new posts

  • reading from multiple files of different format

    Hello All,

    I have a scenario similar to the following example scenario:

    Read a remote XML file with 2 columns with the first column as key.
    Read a local flat file with 4 columns with one column as matcing key from above file.
    Merge records from each file by matching key columns so that the pojo has now 5 properties (4 + 1 one key column common to both files).
    Write the output to a database.

    I am confused as to how the flow/job configuration will be and how will I be reading multiple files of different formats.

    FootballJob exactly matches my requirement. But here I am talking about huge inputs (like 50000 records of 250 columns). So I am skeptic as to whether the approach taken in FootballJob will be efficient.

    Please help!!

    Last edited by nikunj_c; Aug 8th, 2011, 04:00 AM.