Announcement Announcement Module
Collapse
No announcement yet.
OLTP and Batch Co-Existence Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • OLTP and Batch Co-Existence

    Hello every one,
    What is the best way to develop and deploy for co existence of OLTP and Batch. Can I deploy the Business logic Pojo's separately and call them from EJB for OLTP and from Batch container for Batch execution. How database access will be taken care? Do I need to implement data access layer or is it provided by framework for coexistence. Is there any case study/ document for implementation of OLTP and Batch scenario.

  • #2
    Re:

    Hello every one,
    I have gone thruogh the various documents and I felt that the spring batch supports the OLTP and Batch to co-exists. But I am not able to map the level of co-existances. By level of co-existances, I mean that only process (or computational) logic can be common or even DAL (Data acess layer) can be same. I would prefer to do feasibility study on this topic. I want to get the knowledge about the extent to which I can reuse my code. Is it only my process logic will be reused or even my DAL.

    According to me,
    In terms of batch and OLTP, there are number of important considerations.

    1) Data Access and the design of the DAL is one of the most important components to think through. The Data Access Layer (DAL) and more importantly its Data Access Objects (DAO) will be written differently for batch and OLTP. DAO's for batch will use JDBC Batching for example when writing to the database, whereas OLTP DAO's will not. We may do more data pre-fetching in the batch DAO's since data access will probably be sequential versus OLTP where it is random.

    2) Will your batch and OLTP run in the same JVM or run in different JVM's.

    3) If you have a business service, for example to calculate interest on a bank account record, that can be executed from OLTP and from Batch, then your design approach will be slightly different. I describe the design for such a services as "record-oriented services", where the service is written to operate on a business record. That service doesn't (and shouldn't) care about where the record came from, the service only cares about calculating the interest and returning an object with the interest computed.

    I may be incorrect in some way. As I haven't evaluated above mention points via a concreate example. But whatever I tried to conveyed is more of common sense and not rocket science. But I want some of the experience persons (in this spring batch domain) to share thier views.

    I am looking forward for the various approaches, design pattern, designing consideration, implementation issues as well as a concrete case study (if possible).

    Hoping for positive reponse.

    Thanks in advance.

    Comment


    • #3
      All your points are valid. There are definitely sometimes compromises when re-using code between batch and online systems. There was a case study at Spring One this year from a German healthcare provider where they re-used some online data access code. In their case it was using Hibernate so the business code did not need to know about batching or write behind (because Hibernate takes care of that). There is a JIRA open against Spring Core to provide similar services to Spring JDBC clients.

      Comment


      • #4
        It will great help to me if you can provide that JIRA link. I couldnt find the case study on JIRA site.

        Comment


        • #5
          The JIRA link you want is http://jira.springframework.org/browse/SPR-4859. The case study was a conference presentation - there's no reference to it in JIRA.

          Comment


          • #6
            Originally posted by gaurav View Post
            1) Data Access and the design of the DAL is one of the most important components to think through. The Data Access Layer (DAL) and more importantly its Data Access Objects (DAO) will be written differently for batch and OLTP. DAO's for batch will use JDBC Batching for example when writing to the database, whereas OLTP DAO's will not. We may do more data pre-fetching in the batch DAO's since data access will probably be sequential versus OLTP where it is random.

            2) Will your batch and OLTP run in the same JVM or run in different JVM's.

            3) If you have a business service, for example to calculate interest on a bank account record, that can be executed from OLTP and from Batch, then your design approach will be slightly different. I describe the design for such a services as "record-oriented services", where the service is written to operate on a business record. That service doesn't (and shouldn't) care about where the record came from, the service only cares about calculating the interest and returning an object with the interest computed.
            1) You don't necessarily have to do batching. I generally counsel many of my clients to code their DAO's normally, only utilizing batched features if performance issues warrant it. (It will also depend upon how high you can get your commit interval as well)

            2) I would highly recommend that you run batch in a different JVM from from your OLTP processes. OLTP and BATCH applications are going to have completely different needs for things like heap space, etc. Not to mention, you don't want your batch application having an issue to cause your OLTP system to go down as well.

            3) I'm not sure I completely understand what you're saying here. If you code your business service correctly, to be stateless and operate on one 'item' at a time, it should be easy to use the service in both batch and oltp without issue.

            A couple other things you're missing are transaction configuration. In general, batch systems are going to need massively higher transaction timeout settings compared to oltp. The same goes for differences in connection pooling, etc.

            If you private message me your email address, I can send you a copy of the presentation we gave at SpringOne that Dave mentioned.

            Comment


            • #7
              you considered that case
              "If you have a business service, for example to calculate interest on a bank account record, that can be executed from OLTP and from Batch, then your design approach will be slightly different. I describe the design for such a services as "record-oriented services", where the service is written to operate on a business record. That service doesn't (and shouldn't) care about where the record came from, the service only cares about calculating the interest and returning an object with the interest computed."
              In all the examples so far, we have just considered one single business service method which takes pre-fetched domain object as input , does some processing and returns some output.

              in real life applications we have business processing in different service methods and each of these service methods requires different domain object.. how the data (domain objects) should be fetched in this case?

              Comment

              Working...
              X