Announcement Announcement Module
No announcement yet.
Reading Multiple Database Rows Per Item Page Title Module
Move Remove Collapse
Conversation Detail Module
  • Filter
  • Time
  • Show
Clear All
new posts

  • Reading Multiple Database Rows Per Item

    I have an input data set that is composed of multiple rows from the database. It's an end-of-day digest job, where a given user may have many rows scattered throughout the database table. The job has to consolidate those rows into a single item (type Collection, I guess), and it gets written out as a single item.

    I'm wondering how to implement the ItemReader. This seems a lot like the AggregateItemReader, but I don't think that class is appropriate (looks like it's made for file input).

    The only other solution we could come up with was a DrivingQueryItemReader. We could use the KeyGenerator to get a list of all the users to be processed (SELECT DISTINCT(userid) FROM digest). But then the secondary query would be different, and the result set would return multiple rows (SELECT * FROM digest WHERE userid=?) that need to be mapped into one Item of type Collection.

    Any advice on how to implement this?

  • #2
    I would probably start with an ItemReader that returns user ids, and then transform that into the aggregate record before writing it out (you could use an ItemTransformerItemWriter or write your own simple ItemWriter that dies the aggregation and then writes it out, like a normal Dao).

    I didn't think the AggregateItemReader was anything to do with files, as such, but it doesn't look like it fits your use case either.


    • #3
      Would this be a reasonable solution:

      Use a DrivingQueryItemReader with sql="SELECT DISTINCT(userId) FROM digest ORDER BY userId" and also the appropriate restart query.

      Then, create a DelegatingItemReader. The DrivingQueryItemReader would be set as the delegate in the DelegatingItemReader. When read() is called on the DIR, it calls the DQIR which returns the next key. The DIR then calls some other custom SQL and returns the List to its caller.

      The only flaw I see with this is that the DelegatingItemReader doesn't seem to have a protected method to override, like DelegatingItemWriter.doProcess(). I guess I'd have to override the read() method?


      • #4
        Yeah, you could also just make it composite. That is, create an ItemReader and contains another ItemReader. That's why Dave was recommending using the ItemTransformerItemWriter, because in your ItemTransformer you could take the ID and pass it to a dao to get back your list. I suppose we should think about doing the same thing for item reading. Maybe something like ItemTransformerItemReader, wouldn't be too hard.


        • #5
          Ok, thanks for the advice!