Announcement Announcement Module
Collapse
No announcement yet.
GemfireRepository with json-region-autoproxy inop Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • GemfireRepository with json-region-autoproxy inop

    We have a bit of a nightmare with needing to deploy entities to all the gemfire servers. If we update an entity it then needs to moved to all servers and we end up version conflicts occasionally... its a mess. So json would be a great alternative. (Technically so would straight pdx - but I dont see how to do that with repository either.)

    I have the following setup. This works perfectly fine for a normal repository with a legit entity.

    <gfe:client-region id="myregion" cache-ref="myregionCache" shortcut="PROXY"/>
    <gfe-data:repositories base-package="com.garmin.bws.wellness.data.access.repos itory" />
    <gfe-data:json-region-autoproxy pretty-print="false" region-refs="myregion" convert-returned-collections="false"/>

    Anything that returns a collection always returns as arraylist of PdxInstance - not the actual entity I expect. If repository knows how to serialize my entity to json and then to pdx it should be able to go the other way as well.

    findById just throws a ClassCastException.
    java.lang.ClassCastException: java.lang.String cannot be cast to com.garmin.bws.wellness.data.manager.impl.Wellness RegistrationEntity
    at com.garmin.bws.wellness.data.manager.impl.Wellness RegistrationManagerImpl.getRegistration(WellnessRe gistrationManagerImpl.java:123)

    In this case the repository is typed as WellnessRegistrationEntity so it tries to cast the json string.

    Am I trying to do something stupid here? An inappropriate use of repository? Any insight would be helpful. I can maybe come up with a little sample app if that would help.

    Thanks!

  • #2
    @hubbardr1 This will be a multi-part response. However, in short, because you are dealing with "version conflicts" you will want to use PDX since that is exactly what it was designed to handle in addition to portability between language clients (See the GemFire User Guide for further details on PDX Serialization).

    First, lets address why the JSON solution is not working.

    JSON will not work because what needs to go into the Region is JSON and what comes back out is also JSON (where internally, GemFire converts to/from PDX). The Repository abstraction and extension to Spring Data Commons Repositories deals in objects, POJOs, basically your entity classes. It knows nothing about the converting and storing entities as JSON. Thus, using the Repository abstraction with your approach is similar to the following on a 'put'...

    Object (entity) -> JSON -> PDX

    And on a 'get'...

    PDX -> JSON -> Object (entity)

    Except, there is nothing in the Repository abstraction that is going to convert your entity into JSON so that subsequently GemFire will turn the JSON into PDX, and on the way out, GemFire converts the PDX to JSON, but then nothing in the Repository abstract will convert the JSON back into your entity class type.

    Make sense?

    So, you have a few options...
    Last edited by John Blum; May 5th, 2014, 04:49 PM.

    Comment


    • #3
      Well the ReflectionBasedAutoSerializer is doing exactly that right?

      <bean id="pdxSerializer-wellness" class="com.gemstone.gemfire.pdx.ReflectionBasedAut oSerializer">
      <constructor-arg value="com.garmin.bws.wellness.data.manager.impl.W ellnessRegistrationEntity" />
      </bean>

      Entity --> PDX --> CACHESERVER --> PDX -- Entity

      And the repository interface only knows about my entity. But Gemfire only knows pdx.

      With JSON autoproxy (while I understand its a redundant abstraction) could look like this:

      Entity --> JSON --> PDX --> CACHESERVER --> PDX --> JSON --> Entity

      Why would you ever want to do this? Good question. Because the documentation implies that you can? We just have a little data visualization tool that formats json a lot prettier than pdx.

      Anyway - forgot JSON then. How would I convert the existing app to store the PDXInstance vs WellnessRegistrationEntity? Do I have to abandon repository? Ideally I dont want to have to deploy my entity on the server at all...but I still want functions and OQL queries to work.

      Comment


      • #4
        GemFire only knows PDX if your Server's CLASSPATH is not configured with the class type of your domain entity and read-serialized is true (not entirely certain the later is required in a client/server scenario). I do know "GemFire avoids changing data format whenever possible" (based on use) unless de/serialization is required (e.g. Serialization is required for Partitioned/Distributed Regions, Persistence, Overflow, Distribution/Messaging of Data between Peers, Client/Server Communication; De-serialization is required when read-serialized=false and performing a region.get("key") operation would result in getting a domain object back).

        Given your client/server scenario, 1) the domain entity will be serialized when sent to the Server using the configured serialization option, 2) the Server will store the entity in "serialized" form in the Region (thus, not requiring the domain entity class to be on the CLASSPATH), and so 3) ...

        Client (Entity) --> PDX -- CACHESERVER --> PDX --> Client (Entity)

        Is technically correct, assuming, of course, the "Client Cache" is not configured with "read-serialized".

        ~~~

        However, Region auto-proxying, using <gfe-data:json-region-autoproxy>, is specific to Spring Data GemFire. GemFire knows nothing about JSON really in so much that there is no "automatic" conversion from/to JSON/PDX. To store JSON in GemFire using the GemFire API, it is all manual using the JSONFormatter API.

        After digging a bit more, I think I understand your scenario and problem better. While you can...

        Entity ---> JSON --> PDX --> Region (entity stored as PDX)

        You cannot...

        Region (entity stored as PDX) --> PDX --> JSON --> Entity.

        The conversion) between the PDX instance and the Entity is lost in translation. Also, because your proxying the Region to handle JSON, it will convert the PDX instance back to JSON assuming that the client (caller) wanted, or deals in, JSON, not entities (i.e. auto-proxying is not Repository context aware).

        The JSONRegionAdvice does not know to convert the JSON back into the Entity's class type based on the type meta-data from the Repository, or even the value-constraint of the Region (assuming it was set), since the Repository 1) did not communicate the type meta-data to the Aspect (for the Proxied Region) 2) the Repository does not really even know the Region is being proxied to "auto-magically" handle JSON, and 3) there is no API in GemFire supporting a "requested" class type, such as... DomainEntityType domainObject = region.get("key", DomainEntityType.class);

        For instance, if the stored entity is a PdxInstance, the following would fail as well without prior type information even when read-serialized is false...

        Code:
          PdxInstance pdxObj = ...;
        
          region.put("key", pdxObj);
          ...
          DomainEntityType domainObject = region.get("key"); // returns PdxInstance as it is based on original Region API use...


        I suppose more could be done to support this translation in the context of Repositories
        (i.e. using the Region value-constraint, since #1 is not doable because of #3) , but I would argue what you are doing is overkill, since you only need PDX to accomplish "version" support.

        So, I would think (have not tested), you just need the following configuration on the Server...

        Code:
          <gfe:disk-store id="myPdxDiskStore" .../>
         ...
          <gfe:cache pdx-read-serialized="true" pdx-persistent="true" pdx-disk-store="myPdxDiskStore" ... />
         ...
          <gfe:partitioned-region id="AppData" persistent="true".../>


        Note, PDX only needs to be "persistent" if the Server Region is persistent. Also note, I am not entirely certain "pdx-read-serialized" needs to be to true on the Server, but it does not hurt.

        Then, on the Client...

        Code:
           <bean id="autoSerializer" class="com.gemstone.gemfire.pdx.ReflectionBasedAutoSerializer">
            <constructor-arg type="boolean" value="true"/>
            <constructor-arg>
              <list>
                <value>com.garmin.bws.wellness.data.manager.impl.WellnessRegistrationEntity</value>
              </list>
            </constructor-arg>
          </bean>
        
          <gfe:client-cache pdx-serializer="autoSerializer" pdx-read-serialized="false" .../>
        
          <gfe:client-region id="AppData" shortcut="PROXY"/>
        Here, pdx-read-serialized should not be strictly required either since the Region API will be used with Entities in the context of Repositories.

        Can you give this go and let me know.

        Thanks
        Last edited by John Blum; May 6th, 2014, 06:41 PM.

        Comment


        • #5
          Other conditions upon which an "serialized" object (PDX, GemFire DataSerialization, or Java Serialization) may result in the object being deserialized on the "Server" is during Queries, even if issued from the client (which occurs when operations are performed on the objects in the Region, e.g. SELECT u FROM /Users u WHERE u.username.startsWith('bo')) as well as certain Function executions. While OQL can access certain fields on objects in PDX serialized form, ops are a different issue. Likewise, Functions may deserialize the data upon which the Function will operate.

          I was also told that setting the read-serialized on the "Server" is advised.
          Last edited by John Blum; May 7th, 2014, 12:17 PM. Reason: Misspellings

          Comment


          • #6
            John, thanks so much for the detailed responses. I will give this a go tomorrow.

            Comment


            • #7
              If you have additional problems, please post to StackOverflow as the Spring Forums will be migrated to SO.

              Comment

              Working...
              X