Announcement Announcement Module
Collapse
No announcement yet.
Share state between job instances Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • Share state between job instances

    Hi, I've got to create a batch script that fetches data from a remote web service. One of the parameters of the webservice is minTimestamp, which tells the webservice to return only the rows which have been created/updated after minTimestamp.
    The other parameter is batchSize, which tells the service how many rows I want to receive.

    It returns rows of data, in such a way that the timestamp of a row is contained within it:
    PHP Code:
    {
      
    "data""...",
      
    "timestamp" 1234

    After reading <batchSize> rows, in order to get the next batch, the timestamp from the last received row has to be passed to the webservice.
    This job has to be run very often as the webservice produces a lot of data.

    So I was wondering what would be the "right" way to do it Spring Batch? In particular how to store state(timestamp) between job runs and how to handle retries.

    I was thinking of maybe adding two steps around the main read/write step in order to save/fetch the timestamp to/from the database and placing it in the JobExecutionContext. But I was hoping maybe there is an easier builtin solution that I have overlooked.

    Any ideas?

  • #2
    There is nothing built into the framework to persist data across job instances. The best way would be to persist that timestamp somewhere outside (a file, database, etc) so you could retrieve it on the next start.

    Comment


    • #3
      OK, thank you.

      Comment

      Working...
      X