Announcement Announcement Module
Collapse
No announcement yet.
Removing of Step Scope? Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • Removing of Step Scope?

    Dear all,

    I have just saw release of m5 thanks a lot for your great works.

    I found that step scope is removed in m5. I just wonder what if I need to put stateful information in my tasklet/reader/writer? Is prototype scope enough for what I am looking for?

    e.g. I have a job, containing two steps which use the same tasklet, in which I need to keep some counter. And multiple instance of same job will be run concurrently. In the past day I set those taskelt (or reader/writer) to step scope. Shall I set it to prototype now and make use of <aop:scoped-proxy /> in this case now?

    Thanks

    Adrian

  • #2
    Wouldn't prototype scope work for that? It seems like the two steps only need their own copy.

    Comment


    • #3
      Doesn't that mean that the whole chain of beans has to be prototype scoped?
      Since it is enough that the job bean is a singleton that all the the steps will only be injected once. Since several instances of the job will run concurrently, the counter (in the given example) will be overwritten.

      Comment


      • #4
        I agree with that step scope is useful. For example, you have a SystemDate value, which is to indicate the date of the transaction to be submitted to another system. In this case, you want all processing within the step to share the same SystemDate.

        In similar sense, it is also useful to have a job scope. May anyone tell if such scope exists in m5? or is there any way for developers to implement their own scope?

        Comment


        • #5
          I agree that step scope and job scope are useful for lifecycle management of beans. I'm upset that step scope is gone, but you can mimick its functionality using the StepListener class instead.

          For the record, prototype scope does NOT substitute for step scope
          It offers the same net effect for bean creation but prototype scoped bean instances do not get cleaned up until the application context is released, which can cause all sorts of problems. It also won't work if the same bean needs to be passed to multiple collaborators (e.g. item readers, item writers) within the same step -- in step scope they would receive the same bean, in prototype scope they would each get their own copy.


          As for step scope being removed, see http://jira.springframework.org/browse/BATCH-364 -- feel free to add comments if you'd like the issue revisited.
          Last edited by dkaminsky; Mar 6th, 2008, 02:03 PM. Reason: added JIRA details

          Comment


          • #6
            I do not have access to JIRA before. After I registered and logged in, I still cannot find a place (or command button) to add the comment on the issue. Any clues?

            Comment


            • #7
              So what will be the basic proposed way to define jobs in a multi-threaded situation?

              A singleton job, with stateless step artifacts (reader, writer, tasklet) being singleton, and stateful artifacts being prototype scoped + aop:scoped-proxy; Or

              Whole chaing of job, steps, artifacts being prototype scoped?

              Have just retrieved m5 and may need some time to take a look on the new changes. However I believe the changes should make the overall architecture cleaner and easier to understand Thanks for the great works

              Comment


              • #8
                @wordywordy - if you fail to log in then log in correctly, JIRA sometimes does not redirect you to the original page you intended to visit. Use the link in this forum directly in your browser after logging in and see if that works

                @adrianshum - most likely you'll want singletons more than prototypes, with listeners to manage lifecycle needs instead of the step scope. Prototypes should be used if you need factory-like functionality, like getting a new instance of an item reader for several steps within the same job or between jobs defined in the same file.

                Btw, I don't think you should need to use the aop:scoped-proxy tag at all with prototypes. Since they always return a new instance there is no contextual information that would necessitate a scoped proxy.
                Last edited by dkaminsky; Mar 7th, 2008, 12:58 AM. Reason: grammar

                Comment


                • #9
                  Iím struggling with this step scope removal, so I can only assume Iím failing to grasp something fundamental, hereís my problem:

                  In my scenario multiple jobs can be run in the same process (we are using the Quartz Scheduler). We are using instances of the StaxEventItemWriter and StaxEventItemReader. Now that we no longer have a step scope the same instance of the Stax classes are being used again and again. And since its state is not being reset on open() we are getting all sorts of problems when it is using things like its lastCommitPointPosition from the previous job run; assert exceptions are being thrown in the writer and the reader can not longer handle job restarts.

                  With the step scope I would have simply given it a step scope and used aop:scoped-proxy. Now Iím unsure what to do. These classes donít implement StepListener and even if they did Iím not sure how that would help when multiple jobs runs concurrently.

                  Am I missing some simple solution to this problem now step scope has gone?

                  Comment


                  • #10
                    Andy,

                    isn't switching to prototype scope all you need?

                    Comment


                    • #11
                      Prototype scope was indeed all I needed to fix my Stax problems, although it wasn't all that straight forward:

                      1) I was concerned that I was having to push the prototype scope on all parent beans, that included not only the step and job put also various spring Quartz beans that were being injected with by job. Eventual I discovered the prototype ďmadnessĒ could stop at the MethodInvokingJobDetailFactoryBean I using for my Quartz configuration.
                      2) When I made my job bean prototype scope, I was having a problem with the jobConfigurationRegistryBeanPostProcessor which somehow was registering my job more than once and throwing a DuplicateJobException. I suspect this may have been a configuration problem which I have now fixed.

                      It is all working like a dream now, and I have to say, it does seem cleaner after removing all that aop:scoped-proxy stuff.

                      Comment


                      • #12
                        I feel I should add something here since it was my idea to scrap step scope. There are other concerns to discuss around that decision, but the main one for the purposes of this thread is that prototype scope for readers and writers is not intended to be an adequate replacement for step scope (though it may well work in individual cases). In fact, generally there are many parts of a job that are intrinsically stateful, including the Step implementations themselves potentially, by association with other stateful components. While step scope solved that problem, it only solved it if the user understands it and uses it properly. We need a better solution. In the scope of 1.0 the recommendation is to use a new ApplicationContext for each Job execution. The samples contain two demos of this pattern, re-using the launcher and repository configuration as a parent context for all jobs (see TaskExecutorLauncher and QuartzJobLauncher).

                        Comment


                        • #13
                          Using a separate ApplicationContext for each execution as an implicit scope will do the job, but it is a bit constricting in case you're not just running a single job from the command line.

                          For example if I schedule many jobs in a single JVM, some may get queued and can be waiting for execution for a long while. I now need to keep a reference to both the job and the context so I can close the context when the job is finished (unfortunately a JobListener won't do because it doesn't get called when a job throws an exception).

                          I thought about advising the job, but I'm not sure how that will work with restarts and when persisting the job. It's a little clumsy.

                          If I use a step listener to destroy the context then it will be be built for each step in the job which is silly.

                          I think I liked better the separation between job configuration (which is a singleton) and a job state (which is step scoped).

                          Comment


                          • #14
                            Good point about the JobListener, but that is too narrow in scope anyway really - the Job has to exist before the JobListeners can get their callbacks. Advising the Job is the wrong scope as well for the same reason, unless you can get a reference to the ApplicationContext in your advice (I can't think of a nice way to do that).

                            The right level of scope and abstraction is actually JobFactory, and the one that we are using in the samples to demonstrate the ApplicationContext per job execution strategy was missing the close callback for the ApplicationContext. I just fixed it, so you can see a way to do it pretty succinctly in ClassPathXmlApplicationContextJobFactory, if you look in SVN (by checking out the source code or looking in fisheye).

                            Comment


                            • #15
                              I checked out the solution you described from SVN. Actually, it looks very much like my first shot at cracking this. However, i soon replaced it with real AOP since when using delegation you loose access to the underlying Job interface (for example in SimpleJob you can call setListeners).
                              So using an 'After' advice proxy I get the same effect as your solution and more. It still has the problems I mentioned above.

                              It looks to me like the factory needs to return a different class of objects. Perhaps JobContainer or ConfiguredJob or whatever, and that object should be passed around to other collaborators (for example JobLauncher) which will call its getJob() to get the job and execute it, and dispose() method after execution.

                              Comment

                              Working...
                              X