Announcement Announcement Module
No announcement yet.
High volume Transaction Page Title Module
Move Remove Collapse
Conversation Detail Module
  • Filter
  • Time
  • Show
Clear All
new posts

  • High volume Transaction

    I have to read 100000 of rows and process and send them to some other system as message.

    How can i use SB in it ?
    Can SB can be run as standalone jar file ?

    if so, can i get a sample project ?

  • #2
    It's probably a mistake to try and process large numbers of items in a single transaction. You could stage them in a database and then send a message with their primary key range or something equivalent, and that would be quite efficient. Spring Batch could be used to do that, and it is packaged as 2 jar files. If you look in the distro or the source code on github you will find samples and a simple-cli project.


    • #3
      I agree that would be a really big transaction. SB gives you the means to split it in smaller chunks (via configuration) and then you can take any route you want, use a database as intermediary like Dave sugests or publis them to a JMS queue, etc. You can store/send the data you read to pretty much any device/technology if you do a custom ItemWriter to fill your needs


      • #4
        Hi Dave,

        Can you please let me know,the example name ?
        tried to find it,was not able to, find , probably i miss something.
        I need the example which has staging to db as you mentioned.

        Thanks in advance.
        Last edited by fusionobject; Jun 20th, 2011, 11:53 PM.


        • #5
          Sample with staging: