Announcement Announcement Module
Collapse
No announcement yet.
Question on one time file processing and consolidating the data Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • Question on one time file processing and consolidating the data

    Hi All,

    I'm new to spring-batch. I have tried some examples of reading from files and writing to another file after processing. Now I have come across a requirement to read all data from a file and consolidate and then write the data. Is there any specific reader/writer provided by spring-batch. If not how can I achieve this? An example will be highly appreciated.

    Thanks,
    Sanat

    For example lets say I have a file with below data

    client1,operation1,12.90
    client1,operation2,43.90
    client2,operation1,21.00
    client1,operation1,12.43
    client2,operation1,21.30
    client2,operation1,12.40

    now I want to consolidate the data based on client and operation and write a summary as

    client1,operation1,25.33
    client1,operation2,43.90
    client2,operation1,54.70

  • #2
    As per my understanding that is out of scope of spring batch processing... or atleast you need another tool to do it or have some other means to do it.

    One way I could think of is to have a step to write all the records to a database and then have another step with consolidated SQL to read from database and write it to another file.

    Comment


    • #3
      I agree with hailspring's suggestion of using the staging table. The action you are describing is really a query and querying a db table will be a lot easier than trying to query a flat file.

      Comment

      Working...
      X