Announcement Announcement Module
Collapse
No announcement yet.
Using Spring Batch with EJBs in container Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • Using Spring Batch with EJBs in container

    we are trying to evaluate the use of spring batch to be able create a batch job which calls EJB deployed in a container like Weblogic. We would like to leverage chunking in order to avoid transaction timeouts.

    Does this sound like a realistic scenario that we can leverage spring batch for?

    We need to be able to call multiple services, process the retrieved records and perform inserts/updates into the database using other services and commit periodically based on the chunk size.

    any advice would be helpful! Thanks!

  • #2
    You can definitely use Spring Batch within an Application server. But I'm curious whether you want to launch it through an EJB, or launch it some other way, but still call EJBs?

    Either way, your use case of periodically committing based on a chunk size is definitely achievable.

    Comment


    • #3
      The sequence would be
      1. Scheduling software such as Autosys launches an batch process.
      2. The batch process invokes an EJB call which retrieve records to be processed.
      3. Inside the EJB call, a loop is started to process retreived records. Inside the loop, further calls are made to retrieve, create and update other records via EJB calls.

      Does this approach make sense?

      Thanks!

      Comment


      • #4
        Originally posted by BatchWinter View Post
        1. Scheduling software such as Autosys launches an batch process.
        It's a little more complex than that. Autosys will likely call a process (i.e. shell script) that must invoke an ApplicationClient, which can interact with EJB's. This isn't too hard, but I think you'll find that it's not as straightforward as it could be.

        2. The batch process invokes an EJB call which retrieve records to be processed.
        If you're going to have your batch process invoke EJB's, you should make sure that the job itself is running in the container and not an application client, otherwise it would be extremely slow due to serialization. Ignoring that for a moment, what exactly is the EJB returning? Hopefully not all the data to be processed, since that wouldn't really be batch. In order to enable 'periodic committing' your EJB call would need to return one 'item' at a time, similar to the contract of a Spring Batch ItemReader. And honestly, I can't see much of an advantage of calling said EJB. Unless you have to call some legacy code, I would avoid it. And even if it is legacy, I would look at some of our 'out-of-the-box' item readers to see if you could replace the EJB's with them.

        3. Inside the EJB call, a loop is started to process retreived records. Inside the loop, further calls are made to retrieve, create and update other records via EJB calls.
        The way Spring Batch works is to read one item, then write that item. This is much more advantageous than pulling back all of the records, then trying to write them all out one at a time. In this case, your number 3 describes what we could call an 'ItemWriter'. The only difference is that the loop is outside the ItemWriter, and is giving it the items to write, as illustrated by the psuedo code below:

        Code:
        REPEAT(while more input) {
        
        	TX { 
        		REPEAT(size=500) {
        
        			input; 
        			output;
        
        		}
        	}
        
        }
        (It should be noted that the above psuedo code is purely for explanation purposes)

        As you can see, a loop will continually read one item form input, then output it, periodically committing until there is no more input.

        Comment


        • #5
          Thank you so much for the explanation. You are correct. We already have shell script that invokes the ApplicationClient which in turn calls the EJB.

          While the actual work does get done within the container, the main call is done to invoke the job.

          Original implementation is returning a set of Ids to be processed from the database, which you correctly point out is a problem. We should do it one at a time using ItemReader.

          The ItemProcessor then should call the EJB methods to do the processing.

          How should we manage the transactions in this case? After a certain number of records are processed, commit the transaction. Can this be setup to use the Container managed tranasction, so weblogic commits after the commitInterval is reached?

          Thanks again for your excellent insights.

          Comment


          • #6
            You can't use Container Managed Transactions at all. You're right that Weblogic is going to try and commit them when it sees fit, which won't mesh with how Spring Batch wants to commit. You'll need to turn off CMT and give a transaction manager to spring batch, which will control the commits and rollbacks.

            Comment


            • #7
              AutoSys scheduler/Spring Batch application deployed on J2EE application server

              I am looking for guidelines to design a Spring batch enterprise application (running within an application server) using a system scheduler like Autosys as the scheduler/trigger. Basically I am wondering how to best fill the gap between a shell script based scheduler and an enterprise application server like WebSphere/WebLogic? By the way this has to be done thru messaging...

              I understand that this is more an AutoSys (lack of) J2EE connector issue but we're stuck with it.

              Any reference implementation (or examples) regarding this specific architecture?

              Comment


              • #8
                Originally posted by lucasward View Post
                You can't use Container Managed Transactions at all. You're right that Weblogic is going to try and commit them when it sees fit, which won't mesh with how Spring Batch wants to commit. You'll need to turn off CMT and give a transaction manager to spring batch, which will control the commits and rollbacks.
                what if we dont want the spring batch to control the commits??

                Would this be tooo bad?

                Comment


                • #9
                  what if we dont want the spring batch to control the commits??
                  For most people one of the biggest if not the biggest benefit of using Spring is not having to use container managed transactions. Can you explain why you would want to delegate batch commits to an app server?

                  Comment


                  • #10
                    Originally posted by Dave Syer View Post
                    For most people one of the biggest if not the biggest benefit of using Spring is not having to use container managed transactions. Can you explain why you would want to delegate batch commits to an app server?
                    Honestly i am not sure how well it will work.

                    The SLSB i have actually calls out to two remot EJBs (which ironically are on the same app server) (fwiw i DIDNT design this architecture i inherited)

                    I am using Seam and now i know i can pass in the transaction manager back to the Spring one, but i am not sure how that will effect the remote ejbs since they are not seam beans. Honestly i am not even sure if the remote beans are CMT or BMT (suppose i should look that up)

                    Comment


                    • #11
                      I haven't understood your architecture yet. You have a batch job which delegates to an SLSB, which in turn calls a legacy remote EJB? I wouldn't want to count on the remote call belonging to the same transaction as the caller at all in that situation, unless it is optimised into a local call by the container. (Websphere has a monopoly on remote transactions over RMI if I recall correctly, so the container might also save you in the opposite case.) Anyway, maybe you can check the legacy EJB, and if it is deployed with CMT then try and get it re-deployed locally with BMT? Or else make sure that it is idempotent, then you don't really care if it isn't part of the same transaction as your batch job started. I don't think the fact that you are launching a job from a Seam application is relevant as far as Spring Batch is concerned - we just need a normal Spring PlatformTransactionManager, and you should be able to provide one of those quite easily.

                      Comment


                      • #12
                        Oh the reason i mentioned Seam was that with Seam's built in spring integration i could have given you the transaction manager used by the Seam bean thus problem solved.

                        Yeah i'll recheck the Remote EJBs, right now the push at our company is actually to make them all web services (since C++ apps may also need to interact with them) As far as deploying it locally depending on the service that may be a possibility ... i am not sure if it's my groups code or not though.


                        Thanks for oyur help .... i did quite a bit with Spring in the 2.0milestone phases .... but then switched to a company that was sticking with 1.2.

                        Comment

                        Working...
                        X