Announcement Announcement Module
Collapse
No announcement yet.
Skipping files based on lastMod? In a listener? Custom partitioner? Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • Skipping files based on lastMod? In a listener? Custom partitioner?

    Is it possible to skip a step from the beforeStep() method of that step's listener? I am new to Spring Batch, so I may be going at this the wrong way, but here is my situation:

    1) A job kicks off, using a MultiResourcePartioner to hand off every xml file in a directory (resources is configured with /path/to/dir/*.xml

    2) The partitioner hands these files to a step that uses a StaxEventItemReader to parse normally. However, most of these files are in a different character set than what we need, and contain illegal characters. I have a StepListener that fixes these issues before the actual XML processing.

    The problem is that I would like only to parse those xml files in the directory whose lastmodified time is a certain period (say, 5 minutes) before the start time of the execution. The partitioner does not have access to timing, whereas the listener does not appear to have the ability to abort the current step. (Setting the status in the beforeStep method does not prevent the reading/writing from occurring.)

    Is there a strategy I am missing to parse files not only by mask, but also by lastmod?

  • #2
    It sounds to me like you need to modify your partitioner? If you pass all files out to worker steps it's quite inefficient if most of them don't pass your filter.

    Comment

    Working...
    X