Announcement Announcement Module
No announcement yet.
GemFire and database Transaction coordination bug Page Title Module
Move Remove Collapse
Conversation Detail Module
  • Filter
  • Time
  • Show
Clear All
new posts

  • GemFire and database Transaction coordination bug

    I am trying to use GemFire as XA transaction coordinator via Spring data and it does not work

    If everything configured via GemFire and transactions managed programmatically the transaction coordination works. So it looks like there is problem somewhere in the Spring level.
    for version 1.3.3 it is plain impossible to bind data source to JNDI tree.

    For configuration like this:
    <gfe:cache id="gemfireCache" properties-ref="props" copy-on-read="false"
    pdx-serializer-ref="autoPdxSerializer" use-bean-factory-locator="false">
    <gfe:jndi-binding jndi-name="mydb" type="XaPooledDataSource"
    xa-datasource-class="com.mysql.jdbc.jdbc2.optional.MysqlXADataSo urce"
    <gfe:jndi-prop key="URL" type="java.lang.String">jdbc:mysql://localhost:3306/app?autoReconnect=true</gfe:jndi-prop>


    the data source is not available under "java:/mydb"
    1.4.0 fixes this, and now the proper type is XAPooledDataSource

    However transaction coordination fails

    for a method like this:
    public void unSuccessfulMethod() {
    throw new RuntimeException("test exception");

    private void dbAndGFInsert() {
    dataRegion.put(new Integer(1), "a");
    dataRegion.put(new Integer(2), "b");

    jdbcTemplate.update("insert into gemfire(key1,value1) values( 1,'a') ");
    jdbcTemplate.update("insert into gemfire(key1,value1) values( 2,'b') ");

    GemFire changes are rolled back fine, but database changes persist. Please see full reproduction case project attached.

    Any ideas how to fix this problem?

  • #2
    Ooops, this forum does not allow attaching zips or tgz archives


    • #3
      @kgignatyev Just writing to acknowledge that I read this post and I am aware of your problem. I am looking into this and will have an answer for you by EOD or tomorrow. I am also aware that you have been working with Pivotal Field Engineering (Guillermo and others) and will send them a solution in email to forward to you given this forum does not allow uploaded attachments (other than very small image files, argh!).


      • #4
        Thank you John, hope you will find the cause soon. On the side note, I am curious if it is possible to pass an instance of DataSource to the CacheFactoryBean? It looks like for now GemFire wants just config parameters and then takes care of instantiation of the DataSource itself, which is not always desirable.


        • #5
          Hi Konstantin,

          Just following up to let you know I am still working on the issue; I have uncovered a few things, but not with Spring Data GemFire specifically.

          As you are no doubt aware, Spring Data GemFire is mostly a facade around providing convenient abstractions for configuring, initializing and using GemFire within a Spring context with Spring's conventions, patterns and programming model, etc, in addition to augmenting some of GemFire's capabilities (e.g. Functions). However, less apparent is the fact that Spring Data GemFire attempts to strictly use GemFire's public API for implementation purposes, which presents both challenges and limitations, since unfortunately many of GemFire's capabilities are only available, or exposed in it's "internal" classes, which are subject to change.

          Although this is not a Spring Data GemFire bug, there are several significant limitations in GemFire itself, and specifically when "using GemFire as the JTA Transaction Manager", that can cause issues.

          One example is the limitation you identified above... GemFire wants configuration parameters for the DataSource specified with JNDI properties in order to instantiate the DataSource "itself". This is due in part with how GemFire (in a not so JTA-compliant way) coordinates, or syncs transactional resources to a transaction.

          I definitely agree that it is preferable to configure the DataSource using Spring and pass it to the GemFire's transaction management infrastructure for synchronization with the UserTransaction, if only there were a way to pass a DataSource to GemFire through Spring Data GemFire's CacheFactoryBean after the Cache and GemFire's transaction management get initialized, but unfortunately, GemFire does not work this way.

          A few other things to keep in mind as well...

          1. GemFire is not a true XA DataSource; components of Gem's transaction management are only registered as a Synchronization that tie GemFire Cache/Region operation(s) to the JTA transaction using the appropriate callbacks, described in GemFire's User Guide.

          2. GemFire's JTA Transaction Manager implementation is not fully JTA compliant. As such, it might better to use an open source alternative, like JOTM, for local testing purposes when not running full-fledge integration tests inside a container (e.g. WebLogic) in a production-like environment. One advantage to this approach is it would enable you to define your MySQL XA DataSource in Spring as opposed to using GemFire's JNDI properties. GemFire will participate in any external, existing JTA Transaction Management technology in use when configured properly.

          I am pretty close to getting a test using GemFire and MySQL together working correctly when GemFire is used as the JTA Transaction Manager. I just need to iron out a few more missing, important details from GemFire's JTA implementation, like the TransactionSynchronizationRegistry, which is not present.

          As for your MySQL XA DataSource (mydb) not getting bound to GemFire's JNDI context (as java:/mydb) properly... this is due to an actual bug in GemFire's DataSourceFactory class, which I also ran into during my own testing. Inconveniently, you have to define a vendor-specific property, even though this should not be required given all the other attributes of the <jndi-binding> element, even when using GemFire's native cache.xml.

          Essentially, GemFire incorrectly assumes you will define a vendor-specific configuration parameter using <config-property> in native cache.xml, or alternatively when using Spring Data GemFire's XML namespace, the <gfe:jndi-binding> element's <gfe:jndi-prop> elements.

          In other words, I had to specify an arbitrary vendor-specific DataSource property, such as the following...

            <gfe:cache properties-ref="gemfireCacheConfigurationSettings">
              <gfe:jndi-binding jndi-name="gemfiredb"
               <gfe:jndi-prop key="serverName" type="java.lang.String">localhost</gfe:jndi-prop>
               <gfe:jndi-prop key="databaseName" type="java.lang.String">gemfire</gfe:jndi-prop>
          Here I specified both MySqlXADataSource class's serverName and databaseName configuration parameters although only one was needed. Then the DataSource will be bound to GemFire's JNDI context properly.

          More information to follow soon...

          Last edited by John Blum; Mar 17th, 2014, 07:40 PM.


          • #6
            Perhaps JOTM will not work either (although, I have not tried), but per the documentation...

            In section 1.2, concerning "What JOTM is not..."
            • JOTM does not handle JDBCTM, JMSTM or JCATM resources. The only "resources" JOTM handles are XATM resources (as defined by javax.transaction.xa.XAResource).
            I do not think GemFire, or any component of Gem is a true XAResource either, so, according to JOTM, it will not work.
            Last edited by John Blum; Mar 15th, 2014, 04:49 PM.


            • #7
              What version of the MySQL (5.?) and the Connector/J lib (5.1.29 or 5.0.8) are using?
              Last edited by John Blum; Mar 16th, 2014, 07:39 PM.


              • #8
                Hi Konstantin,

                I am sending the Guillermo and our Community Engineer (Luke) involved in this matter, the solution. They will forward the solution onto you.



                • #9

                  I am following up again because it would seem my solution was "lost in translation" and you never received it. I was just pinged by another GemFire support engineer (Akihiro) again today regarding this problem, and the corresponding support ticket (SR #14452244403).

                  He did share the example test code you wrote containing the missing pieces of the puzzle (namely, your gf-xa-transaction-manager.xml SDG XML config file) that is clearly essential to properly analyze the problem. I did not previously have your example test code, but was able to successfully extrapolate and solve the problem anyway.

                  I even ran into some of the same initial problems you encountered (e.g. no DataSource bound in the JNDI context, which did have a solution as I described above) so knew I was on the right track. Anyway, let me shed some light on your problem.

                  First of all, it is important to know that there are 2 types of transaction management "strategies" in GemFire, as explained in the GemFire User Guide. First is GemFire's Cache Transaction management, which is not JTA-based. The other transaction management strategy in GemFire is simple, though partially baked implementation of JTA supporting global transactions.

                  As you no doubt already know, you need to configure a proper Spring PlatformTransactionManager to support JTA-based Global Transactions. Specifically, you need to use the org.springframework.transaction.jta.JtaTransaction Manager.

                  Also, when the JtaTransactionManager is used in a global transaction "enabled" context (e.g. an application server like WebLogic) it is able to properly identify the transaction manager (JTA provider) to use and delegate "global" transactions to that manager (i.e. WebLogicJtaTransactionManager). This is based on the fact that the "transaction managers/strategy" are published to well known locations in the JNDI context.

                  However, in certain JNDI contexts (like GemFire), the locations are not "well-known". However, you can still explicitly configure the JtaTransactionManager accordingly.

                  So, in your configuration (gf-xa-transaction-manager.xml), you have configure GemFire's Cache Transactions (management strategy) with the use of...

                   <gfe:transaction-manager id="xa-tx-manager" cache-ref="gemfireCache" copy-on-read="true"/>
                  This is not a JTA-aware/enabled Spring PlatformTransactionManager, and it only pertains to GemFire "Cache" Transactions, not to be used for other external transactional resources (e.g. MySQL) too.

                  Therefore, you need to define the following in your Spring config meta-data, as I have done in my example test code...

                  <util:properties id="jndiConfigurationSettings">
                    <prop key="#{T(javax.naming.Context).INITIAL_CONTEXT_FACTORY}">
                     com.gemstone.gemfire.internal.jndi.InitialContextF actoryImpl
                  <bean id="txManager" class="org.springframework.transaction.jta.JtaTransactionManager">
                    <property name="jndiEnvironment" ref="jndiConfigurationSettings"/>
                    <property name="transactionManagerName" value="java:/TransactionManager"/>
                    <property name="userTransactionName" value="java:/UserTransaction"/>
                  Don't worry about the missing JTA interface implementation of TransactionSynchronizatonRegistry from the GemFire JNDI context. Spring seems to compensate for this.

                  Interestingly, if I remember correctly, I think Spring's JtaTransactionManager actually looks for a JTA TransactionManager @ java:/TransactionManager and UserTransaction @ java:/UserTransactions, but for some reason (don't remember off hand) I had to configure these manually.

                  Anyway, the above configuration does work correctly and both the GemFire and MySQL transaction (when using MySQL's MysqlXaDataSource) are rolled back successfully on an unchecked Exception (by default), as my test code demonstrates.

                  If you do not receive my example from support, please feel free to email me directly (jblum at gopivotal dot com), and I will send you my example test code.

                  Last edited by John Blum; Apr 2nd, 2014, 01:57 PM.


                  • kgignatyev
                    kgignatyev commented
                    Editing a comment
                    Thank you, made the changes and got my repro case code working.

                    I know it is a separate issue, but still wondering: will it be someday possible to point and use 'standard' dataSource configured with Spring rather than let GemFire to create and configure dataSource instance?

                • #10
                  Well, the Data Source definition is not specifically an issue with Spring Data GemFire either, but rather a limitation explicitly imposed by GemFire by way of its own configuration with the JNDI properties for "declaring" any other external JTA transactional resource (e.g. the MySQL DataSource... MysqlXaDataSource) used with GemFire.

                  I.e. there is no way to pass a configured DataSource instance to GemFire (either by using GemFire's public Java API or it's native cache.xml configuration). For instance, configuring the pure GemFire transactional example (as I did in my own example test code in addition to the combined SDG/Spring and GemFire example )...

                  <?xml version="1.0"?>
                  <!DOCTYPE cache PUBLIC  "-//GemStone Systems, Inc.//GemFire Declarative Caching 7.0//EN"
                  <cache lock-lease="120" lock-timeout="60" search-timeout="300">
                    <region name="Gemstones" refid="REPLICATE">
                      <region-attributes concurrency-level="16" ignore-jta="false" initial-capacity="101" load-factor="0.85">
                      <jndi-binding jndi-name="gemfiredb"
                  In order for GemFire's JTA-based Global Transactional solution/strategy to function properly, GemFire likes to be in "control".

                  Furthermore, this is only a problem if you prefer to use "GemFire's" own implementation of the JTA Transaction Manager (mapped in the JNDI context at java:/TransactionManager) delegated to by the Spring JtaTransactionManager(the provider-based approach) as demonstrated in my example test code configuration.

                  In fact, I highly recommend that you NOT use of GemFire's JTA TransactionManager in production given it is not a fully compliant JTA implementation. Using it for basic integration testing purposes and app validation as you have done should be fine, however.

                  If you prefer to configure the DataSources separately (I certainly do), or even have more than just 1 other external JTA-compliant transactional resource (like a Message Queue and 1 or more other RDBMS with GemFire), then you have to use a separate JTA transaction management strategy, such as JOTM (or equivalent) anyway, especially when not in an application server environment, as I stated before.

                  Or, if you do run your integration tests in a production-like environment (e.g. WebLogic) then you can just configure the Spring JtaTransactionManager without any additional configuration, or use the WebLogicJtaTransactionManager directly, and GemFire will participate in the Global Transaction as explained in this section of the GemFire User Guide.

                  Either way, you have options. But, this is really beyond the control of SDG unfortunately.