Announcement Announcement Module
No announcement yet.
MapReduce like operations in SpringBatch step Page Title Module
Move Remove Collapse
Conversation Detail Module
  • Filter
  • Time
  • Show
Clear All
new posts

  • MapReduce like operations in SpringBatch step

    Is it possible to do MapReduce style operations in SpringBatch?

    I have two steps in my batch job. The first step calculates average. The second step compares each value with average to determine another value.

    For example, Lets say i have a huge database of Student scores. The first step calculates average score in each course/exam. The second step compares individual scores with average to determine grade based on some simple rule:
    1. A if a student scores above average
    2. B if a student score is Average
    3. C if the student scores below average

    Currently my first step is a Sql which selects average and writes it to a table. Second step is a Sql which joins average scores with individual scores and uses a Processor to implement the rule.

    There are similar aggregation functions like avg, min used a lot in Steps and I'd really prefer if this can be done in Processors keeping the Sqls as simple as possible. Is there any way to write a Processor which aggregates results across multiple rows based on a grouping criteria and then Write Average/Min to the Output table once?