Announcement Announcement Module
Collapse
No announcement yet.
increasing duration between steps over a long period Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • increasing duration between steps over a long period

    Dear all,

    my job does some file analysis over a quite long period of time. There are ~1200 files to read and analyse.
    Within the job specification, I do iteration over a set of 10 steps required to analyse a single file.

    Now I am facing a increased duration on analyzing a file. At start a single iteration over 10 steps took ~ 10 seconds, on half-time the iteration took ~7 minutes. While the execution time of each step approximately constant, the time between ending a step and starting the next one increased from 500 ms to 11 seconds!!

    A memory issue?
    Did anyone experience similar behaviour?

    Require code? Please let me know!
    Thank you very much in advance for your help!!


    Penny_Wise

  • #2
    ... perhaps we found the reason: STEP_EXECUTION_CONTEXT (repository table)

    We do some logging into database tables, and access of STEP_EXECUTION_CONTEXT got quite slow. After truncating the table, spring batch got fast as light again.

    Comment


    • #3
      Did you do any research into what the query was that was the bottle neck? What database type? unless the table had gotten huge, I'd expect it to have a nominal impact at best...

      Comment


      • #4
        Our db expert mentioned the clob as possible bottleneck.
        The db is an oracle 11g. In fact, I generated ~16k rows in that table. Using Toad for displaying the data, we experienced slow performance, too. So from my point of view this isn't a spring batch specific issue.

        Nevertheless I would appreciate features concerning a configuration of logging-depth. I quickly scanned the docs today, but wasn't lucky, though. Is there any possibility? THANKS!

        Comment

        Working...
        X