Announcement Announcement Module
Collapse
No announcement yet.
Architecture advice involving multiple projects/spring context files? Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • Architecture advice involving multiple projects/spring context files?

    I am developing an application that in essence is built out of many smaller but isolated applications, with one web project tying them all together and I am debating about the architecture of my eclipse projects and context files.

    Currently I have (and have traditionally used) an ApplicationFramework and ApplicationWeb project. Service/Dao/Domain classes reside in the application framework along with the service/dao context files so they can be unit tested and isolated from the web.

    In the web project web.xml, I refer to the context files from the framework project and also any context files the web might be using.

    This has worked well in the past, but I am noticing that since I will have multiple smaller projects, each with their own services and domain objects, it may make sense to divide them into their own projects as follows:

    ApplicationFramework
    / \
    SomeAppFramework AnotherAppFramework

    With ApplicationWeb referring to all of them. This would of course mean each subproject has its own context files, which may bring a new set of issues, such as do I then need multiple cache managers to configure the application caches separately and would that be inefficient (or more efficient?), how would the ehcache MBean handle multiple managers etc.

    The alternative is to have the code reside in different packages within the main application framework, although I have a feeling that will grow very fast. Some of these smaller applications will not have much code, some may have a lot more, although I expect them all to have persistent objects of some kind.

    From an architecture point of view I believe there is value in isolating the subprojects, and typing this out I realize my only big hang up is the multiple cachemanagers, so I will look into that to start. But I was wondering if anyone had any high level insight they could share.

  • #2
    Another problem I just realized with multiple context files is the session factory mappings. I don't know how it would be possible to share one session factory without some sort of coupling between the individual dao objects that needed to be mapped.

    It seems like I almost need a centralized place for the context files, and I fiddled with the idea of an ApplicationContext project which basically holds all the context files, and I suppose every project would refer to it when unit testing, and the web when running the application. I don't know though as I've never heard anything about this and it seems hacky by my initial impression.

    Another sort of hack would be to extend the local session factory bean to maybe search for a directory of mappings thats specified in the xml, and also any sub packages to pull in all of the mappings of the individual applications..

    Comment


    • #3
      1) Keep your "business" components outside the webapp module; I understand you are doing that already.

      2) Package your business components by "business domain"/functionality - NOT by class type!! This is very important. Most sample applications and books do it otherwise, and that's unfortunate. Packaging your classes by class type (e.g. "services", "persistence", "entities", etc.) automatically ties all your modules together and will force you to always load the whole project with all source modules - even if you only are working on one particular functional domain. Your complete project should be an assembly of multiple modules each of which should be available individually (as a separate project.) In other words, you should be able, if necessary, to create a smaller workspace with any single source module (or only a sub-set of modules), while all other dependencies are pulled as jars. I see horrendously packaged projects every day - sometimes with thousands of classes - that can't really be split into smaller manageable sub-projects. Don't make the same mistake - especially if you have an opportunity to start from a clean slate. Within each "functional domain" module you should consolidate all things related to this functional domain: entities/model, service API, utilities, etc. So your complete system might look like this:

      common
      -entities (entities reusable by all)
      -service[-related stuff such as common interfaces, etc.]
      -common utils (that may be used by any module)
      - some common constants and exceptions (most should be specific to the related functionality though...)
      user
      - user model (entities)
      - user service API, optional implementation (other implementations may reside in apps)
      - user utils
      - etc.
      bookings
      - bookings model (entities)
      - bookings service API, optional implementation (other implementations may reside in apps)
      - bookings utils
      - etc.
      awards
      - similar to the above
      etc.
      webapp
      -css
      -js
      -WEB-INF
      - config/...
      - views
      - etc.
      web.xml

      Carefully define uni-directional dependencies so that each module has a dependency on "common" and the minimum other modules. Your webapp will have the dependencies on all the above. If you are using Maven, you can create a parent POM for your "business" components, so you can build them all separately from your webapp. You should be able to load each POM into your IDE separately/individually and work on that small project without having to load everything else.

      You may still use a top "parent" POM where you define the versions and global properties. In that case, all you will need to load a small sub-project is the parent POM and the POM for that individual module.

      3) Your webapp should define your Spring configuration for you business modules. I don't recommend putting any Spring config files into your business modules. Configuration is application specific. Although it is absolutely ok to have your business components to have Spring dependencies (nothing wrong with using Spring libs for various purposes, obviously), leave it up to your application to wire things together the way it needs it. Theoretically, another application may reuse the business/common components but provide different implementation of some of the collaborating beans, etc. For unit testing, I personally prefer not to use Spring contexts. Spring-wiring beans for unit tests makes it integration testing rather than unit testing. Wiring your Spring configuration (or stuff like log4j configuration) into the business modules makes them less reusable, and often just adds clutter and confusion. I often see projects where developers have a hard time telling which of the multiple configuration files is actually being used and when. They mistakenly modify the wrong files expecting results while the changes don't take effect... Keep things simple: consolidate your app configuration, e.g. under the webapp/WEB-INF/config/app-context directory, or something like that. I prefer to split configuration into multiple files - also based on the functionality and functional domains.

      HTH,
      Constantine

      Comment


      • #4
        Thanks there is a lot of useful information in your post.

        One thing I am guilty of is Package-By-Layer rather than Package-By-Feature. I know that feature is the winner, its just a hard transition for some reason (perhaps the OCD in me). However I will make the change for this project since I have a clean slate.

        One thing I am still struggling with is the testing. For example, I keep the application context in the business project because I often need a Spring context to test a unit a code. For example, a Hibernate call using Criteria objects is something I most certainly need to test as a unit, but I can't do so without the dao which extends the hibernate template which needs a session factory and ultimately needs to pull in a lot from a spring configuration to run.
        It seems that a unit test turns into an integration test.

        And theres a side benefit of knowing my application context is wired up and running correctly since the web app is just referring to the one I am testing. However I can see the point that the client should be the one configuring, since it is indeed the application's responsibility.

        So I'm still at a loss of the correct approach, because while I would love to have my small application modules isolated, testable, i'm not sure how to go about doing it.

        Comment


        • #5
          Originally posted by Fatefree View Post
          One thing I am still struggling with is the testing. For example, I keep the application context in the business project because I often need a Spring context to test a unit a code. For example, a Hibernate call using Criteria objects is something I most certainly need to test as a unit, but I can't do so without the dao which extends the hibernate template which needs a session factory and ultimately needs to pull in a lot from a spring configuration to run.
          It seems that a unit test turns into an integration test.

          And theres a side benefit of knowing my application context is wired up and running correctly since the web app is just referring to the one I am testing. However I can see the point that the client should be the one configuring, since it is indeed the application's responsibility.

          So I'm still at a loss of the correct approach, because while I would love to have my small application modules isolated, testable, i'm not sure how to go about doing it.
          Not a problem... Just put your test Spring config files into the Resources folder under the test tree, not main. I am assuming you are using the Maven standard directory structure with two distinct source trees: main and test. If not, use it. It will solve your problem. Put any config files you want under the test tree, in "resources". They will be invisible to your "main" source code.

          Comment


          • #6
            Hmmmm. I had originally put the files in the src/main/resources, and was operating under the mindset that the config files should also be shared amongst the modules. I think thats whats causing my problems.

            So in essence, to solve this, every module will have their own set of config files, hidden away in the src/test/resources folder and configured just small enough to get the tests running.

            So now the modules are independent and testable, and ill switch to the new package structure to sweeten the deal.

            Then the web application has a separate set of config files to wire up the app accordingly. I think I'm starting to see the big picture.

            So as an almost unrelated topic, where might code go that is responsible for convenience methods that say, might clear your entire db, or put sample data for you app to use? Would you consider that test classes, or some sort of external support project?

            Comment


            • #7
              I think you got it!

              Originally posted by Fatefree View Post
              So as an almost unrelated topic, where might code go that is responsible for convenience methods that say, might clear your entire db, or put sample data for you app to use? Would you consider that test classes, or some sort of external support project?
              That sounds to me like stuff totally unrelated to you application. So, i would keep that in a separate "maintenance-utilities" module.

              Also, you may want to have some utility-type classes in your test tree that would help you generate some test data. Although, as I said, personally, I am very much against doing any kind of integration stuff or messing with the database in JUnit tests. Perhaps, for integration tests, you could create a special module - that would mock your actual app but without the UI, etc. and have all the required dependencies on your business modules. You could add such maintenance classes/methods to such mock client, run the DB setup before running your integration tests, and then run the cleanup methods. But I would not mix such code with the application or business components themselves. For me, separation of concerns and minimizing complexity are the two most important principles that have never let me down. You can never go wrong with these.

              Comment


              • #8
                One final question if you will. Do you consider the hibernate hbm files to be web configuration as well?

                I can see it either way, where some clients may use different table names or perhaps have different columns required in their app than others and thus should define them itself.

                And of course the other side of the argument is where it would be nice to have the framework components define them and test them once and for all. But perhaps that is the job of some theoretical 'integration' module. What are you thoughts?

                Comment


                • #9
                  Originally posted by Fatefree View Post
                  One final question if you will. Do you consider the hibernate hbm files to be web configuration as well? I can see it either way, where some clients may use different table names or perhaps have different columns required in their app than others and thus should define them itself.
                  Not "web" (obviously), but certainly application configuration. You basically answered your own question: different applications may not only [be allowed to] use different column mappings, but also work with different database schemas, different data sources, and/or different data access implementations altogether - depending on their needs, specifics, and deployment environments. Although it is a very common practice and convenient shortcut to wire in the persistence mappings into the potentially otherwise reusable business components, and even into the domain model (e.g. JPA annotations), I think that is substandard architecture that relies on the very lame thesis: we are not going to reuse anything anyway. Lame... Actually, more often than not, teams find themselves eventually having to implement another app, and another - within the same enterprise. So what do they do? They just duplicate their domain models and business components over and over - instead of reusing what could have been easily reused if they hadn't wired app-specific stuff into those components. I see this all the time.

                  And of course the other side of the argument is where it would be nice to have the framework components define them and test them once and for all. But perhaps that is the job of some theoretical 'integration' module. What are you thoughts?
                  I think you know what I think. I prefer independent components that have minimum dependencies. To me it is mind-boggling to suggest that an architectural decision could be made based on where some developer decided to place his/her test implementations. The only tests that belong with a module are unit tests. That is, the tests that test only the units implemented within the module - without their dependencies/collaborators. That's what mock objects are for. Integration tests should never affect your packaging or architecture. They should live outside and test the modules that know nothing about their existence. Most importantly, there is no such thing as integration testing "once and for all" since the modular design of your architecture and components implies that your components should be usable in various configurations/scenarios/deployments, not just one - hard-wired into the module itself. Hope this makes sense...

                  Good luck,
                  C

                  Comment


                  • #10
                    Thanks again constv, I completely agree with your points about reusable business components. I shy away from annotations for that very reason.

                    For me, the line always got blurry depending on which module I was working with. For example, a CommonFramework module may have reusable domain objects for any application, and as such I fully agree that it makes no sense to expose any configuration files, nor would I have the desire to run integration tests there.

                    But if I am building a specific web application, say a PetClinic application, I might have a PetClinicFramework, a PetClinicAdminFramework, and a PetClinicWeb. From there I have been tempted to both put the configuration/hbm/integration tests directly in PetClinicFramework, since the web application is really the only feasible application that would use them.

                    However I see the argument that if a PetClinicGui were to be written, I may have a different configuration, and in that sense it would be useful to reuse the framework with a new set of config files. I suppose thats where the voice in my head might say, will you really reuse the framework modules that were written specifically for a web application?

                    But I intend to do the correct thing architecturally, so I will move the configuration files to the PetClinicWeb and run the integration tests there. I hope this thread was useful for anyone else experiencing similar design decisions.

                    Comment


                    • #11
                      Fatefree,
                      just the fact that you are so carefully considering all these things tells me that you will do the right thing -- whatever it may be for your particular case. Thoughtful approach based on a carefully designed strategy is the essence of any good architecture and successful project. Thoughtless mimicking someone else's patterns - just "because everybody does it" - is a recipe for failure.

                      Consider the following approach that should allow you to reuse your context-specific configuration between multiple applications without duplicating it in each application - while still keeping your business modules architecturally pure.

                      1. Package your business modules as I described above in this thread - without any configuration, to make them truly reusable and self-contained.
                      2. If you want multiple applications to be able to use the same context configuration (same service implementations, same data source, same data access implementations/hbm configuration, etc.) - without duplicating the config files - wrap each of these "pure" business components into a module that depends on the relevant business components (as jars) and adds configuration - in the resource folder. Obviously, such module would also be a perfect place to keep your specific implementations of services, daos, etc. that are meant to be reused by all those applications that reuse the same deployment context configuration. (The service/domain API should be, of course, defined in your "pure" business modules.)
                      3. Include the latter (modules with configuration) into your applications as dependencies - not the "pure" modules. (Similar to what you would do with objects when you extract generic functionality into a class and create a "wrapper" class that includes the generic functionality by composition.)

                      This way, you will have your business components available for potential reuse in any contexts, as well as more context-specific modules that are composed of the "pure" business modules plus context-specific configuration.

                      HTH,
                      Constantine
                      Last edited by constv; May 21st, 2010, 08:44 AM.

                      Comment


                      • #12
                        Thanks Constantine,

                        That was exactly in line with what I was thinking of in my second post, about the context project (minus all projects referring to it - the other way around now). Back then, which seems like a long time ago, I couldn't grasp the value of it without understanding the bigger picture.

                        But its clear to me now, and thats the approach I will take. As a bonus my web project will only have to concern itself with any web tests rather than integration tests, which sits much better with me. And thanks again, I appreciate all of your advice.

                        Comment


                        • #13
                          Fatefree,

                          This is a good discussion because it highlights many of the problems that come up when you try to break up a substantial real world Spring application into reusable chunks.

                          Something you might like to take a look at is Impala, which provides a lot of support for addressing the types of issues you and Constantine are raising, as well as some simple to use techniques for making parts of a larger application easier to mix and match in different environments and configurations.

                          Cheers,

                          Phil Zoio
                          Impala - modular productivity for Spring applications
                          http://www.impalaframework.org/
                          http://impalablog.blogspot.com/

                          Comment


                          • #14
                            Hi,

                            Interesting thread. I must accept that I face the same concerns regarding the design of a new spring based application. For the moment, I use a layered architecture (Web / Service / DAO / Model) because, first of all, as ConstV said, it’s a commune practice.
                            I would really want to understand where to use a layered and when to use a functional decomposition.
                            I take your example and I create 5 independent maven projects
                            - common-prj
                            - user-prj
                            - bookings-prj
                            - awards-prj
                            - web-prj
                            - pom.xml
                            In my user-prj I will create com.app.user.model.User (+ service, DAO to handle the User). Let’s say that a User can have a List<Award>. To avoid some horizontal dependency I must create the Award in com.app.common.model ? (You’ve said: “common : entities (entities reusable by all)”). Isn’t this a very permissive architecture? The fact that two functional modules can reference an entity, can be enough to qualify it as a commune entity?

                            “I don't recommend putting any Spring config files into your business modules.”
                            Adding AOP your business module implies the writing of some Spring configuration files. Aspects can change (or add functionality to) your business logic, so they have their place (in my opinion) in the same project. I will put the Spring beans configuration in the same project, too. Simply because that, when I define a bean, I need to put the full class name, class which is declared in the same project, and if I rename this class I will see a validation error in my src/main/resource/my-sub-prj-config.xml (the bean initialisation can change, too, imagine that you have to make some attribute as Required).

                            Please don’t get me wrong, I really want to understand when, why and how to configure a Spring – Maven multi module (multi projects) based application using "Package-By-Feature" approach.

                            Thanks,
                            Ivat

                            Comment


                            • #15
                              Well just to answer quickly, for your example I would (and have done, for many similar things) keep User in the common project, and create a new project that depends on common to hold your Awards logic.

                              Then instead of having a List<Awards> in your User object, I would instead attach a User reference to an Award object. Then you can simply query Awards where User = some user id, and get a list of awards for a particular user. This way the actual user object doesn't need to know anything about awards, but your award service can still provide a list of awards for any user. And of course this could apply for any object that needs ownership by a user.

                              Comment

                              Working...
                              X