Announcement Announcement Module
Collapse
No announcement yet.
Spring Data Graph: Map pairs as node properties? Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • Spring Data Graph: Map pairs as node properties?

    Hello graphistas

    We get json data back from a web app and we'd like to store the relevant parts of it in a @NodeEntity. We are currently following the approach to flatten the deep JSON structure to a flat Map<String, Object>, where the keys are converted to a dotted notation.
    Code:
    {
      "foo" : {
        "bar": 1,
        "baz": 2
      }
    }
    is converted to a Map like
    Code:
    "foo.bar" => 1
    "foo.baz" => 2
    in an entity like
    PHP Code:
    @NodeEntity
    public class MyEntity {
      private 
    Map<StringObjectpropertiesToStore;
      ...


    We now would like to store the key-value pairs of this propertiesToStore field directly as node properties. Any ideas how this could be implemented with Spring Data Graph?

    Or are there even simpler or alternative approaches to store a JSON structure in a node with Spring Data Graph without using a Map as a field?


    Best regards,
    James

  • #2
    Right now there is no native support for storing maps in neo4j. You could register a spring converter that converts it to a json string and back again (or to another serialized form).

    There is also the possibility to store the map directly as properties of the node? Perhaps prefixed with "foo." as in your example.

    You might want to look into the source code of spring data graph and field-accessor-factories. Those allow you to register custom handlers for fields (like a Map<String, Object>) and handling write and read operations to this field.

    It would be really cool if you could try that. If you need any hints just ping me. Probably create a JIRA ticket for that and let's continue to discuss there.

    Thanks

    Michael

    Comment


    • #3
      I'll check out the field-accessor-factories you mentioned as this sounds very promising. I'll let you know how I'm proceeding with it.

      Comment


      • #4
        Cool, looking forward to your solution. For the storage of the individual properties it might be useful to reuse PropertyFieldAccessor and/or ConvertingNodePropertyFieldAccessor probably just extract their functionality to a separate, reusable class.

        Comment


        • #5
          I could not find a way to register an own FieldAccessorFactory implementation. I opened this JIRA ticket for that.

          Just an idea...
          I maybe useful to have a Map<String, Object> property be bound to its own referenced node. So maybe the following...

          PHP Code:
          @NodeEntity
          public class Person {
              private 
          String name;

              @
          RelatedTo(type="has")
              
          Map<StringObjectaddress;
          }

          person.address.put("ZIP"8000);
          person.address.put("City""Zürich"); 

          ...could create two nodes:
          [Person]---(has)--->[address]

          where the person node had a "name" property and the address node has the properties "ZIP" and "City" from the map.

          This would keep something of the dynamic nature of neo4j.

          The background of this: We must persist a lot of graphical information of entities that is not needed in the business layer, but must only be persisted and reached back to the UI again. This graphical information is part of a json document with quite a complex structure. Now instead of denormalizing this structure and creating many (from the backend point of view) useless entity classes, we would like to put this information dynamically in a flat map. This map would be stored ideally in its own node.

          What do you think about such a dynamic component in SDG? Could there be other approaches to store key/values dynamically?

          Comment


          • #6
            Hi,

            you should be able to "register" FieldAccessorFactories by overriding NodeDelegatingFieldAccessorFactory and override the method createAccessorFactories.

            But probably there should be a simpler way to register those, I'm not sure.

            Interesting to use @RelatedTo for that, I'm a bit leaning towards a custom annotation? (Or probably none).

            I've also thought about the solution you're proposing.

            What one could do is to have the default-inlined solution for normal Maps and when using @RelatedTo then it is a separate node.

            I'd like to have such a dynamic approach being supported in SDG.

            I'm not sure about using a generic Map structure. Probably just using a neo4j Node or a SDG-MapNode or a simple PropertyContainer would be enough? For a Map you have to support all the operations of the Map interface some of which don't fit so well in this context (and it would have to be a managed Map anyway whose content must be reflected in the graph on any change)

            Michael

            Comment


            • #7
              I just hacked together a proof of concept implementation that stores all key/value pairs of a PropertyMap field of a NodeEntity to the underlying neo4j node. PropertyMap is an newly introduced PropertyContainer like interface (see below). A own FieldAccessorFactory handles getting and setting this field.

              Some observations:
              you should be able to "register" FieldAccessorFactories by overriding NodeDelegatingFieldAccessorFactory and override the method createAccessorFactories.

              But probably there should be a simpler way to register those, I'm not sure.
              Some kind of FieldAccessorFactory registry would be great. I now ended up configuring the whole Spring Data Graph stuff in XML bean notation, because the java config of course does not know anything about my extended NodeDelegatingFieldAccessorFactory and it has to be wired in manually. May there be simpler alternatives to just "replace" the NodeDelegatingFieldAccessorFactory bean with an own implementation?
              Interesting to use @RelatedTo for that, I'm a bit leaning towards a custom annotation? (Or probably none).

              I've also thought about the solution you're proposing.

              What one could do is to have the default-inlined solution for normal Maps and when using @RelatedTo then it is a separate node.
              The problem with the inlined map is that this map does not know which properties belong to the map and which belong to the entity itself, when it is loaded. Imagine the following entity:

              PHP Code:
              @NodeEntity
              class Person {
                  
              String name;
                  
              PropertyMap properties;

              The properties map contains the keys "foo" and "bar". The node then contains the properties "name", "foo" and "bar". When loading the node into the entity, the FieldAccessor for the properties map does not know that the property "name" does not belong to this map, but to the entity itself. I could not quickly find a way to let a FieldAccessor know about the already handled fields. Any ideas?

              I'd like to have such a dynamic approach being supported in SDG.

              I'm not sure about using a generic Map structure. Probably just using a neo4j Node or a SDG-MapNode or a simple PropertyContainer would be enough? For a Map you have to support all the operations of the Map interface some of which don't fit so well in this context (and it would have to be a managed Map anyway whose content must be reflected in the graph on any change)
              I introduced an interface PropertyMap that is the same as PropertyContainer, but without the getGraphDatabase() method. An implementation of this PropertyMap uses a Map<String, Object> as a delegate to actually store the key/value pairs and is used in the DynamicPropertiesFieldAccessor that handles properties of type PropertyMap.

              My next step will be to try to store the dynamic map in a separate node so that I do not have to know which keys are already handled by other FieldAccessors on the entity itself.


              Best regards,
              James

              Comment


              • #8
                Originally posted by ractive View Post
                Some kind of FieldAccessorFactory registry would be great. I now ended up configuring the whole Spring Data Graph stuff in XML bean notation, because the java config of course does not know anything about my extended NodeDelegatingFieldAccessorFactory and it has to be wired in manually. May there be simpler alternatives to just "replace" the NodeDelegatingFieldAccessorFactory bean with an own implementation?
                Ohh... Extending the Neo4jConfiguration and overriding the nodeEntityStateFactory() method is enough to just re-define this bean.

                PHP Code:
                @Configuration
                public class ExtendedNeo4jConfiguration extends Neo4jConfiguration {
                    @
                Override
                    
                @Bean
                    
                public NodeEntityStateFactory nodeEntityStateFactory() throws Exception {
                        final 
                GraphDatabaseContext graphDatabaseContext graphDatabaseContext();
                        final 
                DirectGraphRepositoryFactory graphRepositoryFactory directGraphRepositoryFactory();

                        
                NodeEntityStateFactory entityStateFactory = new NodeEntityStateFactory();
                        
                entityStateFactory.setGraphDatabaseContext(graphDatabaseContext);
                        
                entityStateFactory.setEntityManagerFactory(getEntityManagerFactory());
                        final 
                ExtendedNodeDelegatingFieldAccessorFactory nodeDelegatingFieldAccessorFactory = new ExtendedNodeDelegatingFieldAccessorFactory(
                                
                graphDatabaseContext);
                        
                entityStateFactory.setNodeDelegatingFieldAccessorFactory(nodeDelegatingFieldAccessorFactory);
                        return 
                entityStateFactory;
                    }

                So this actually is enough to wire in my "ExtendedNodeDelegatingFieldAccessorFactory". I never worked with java config before, but this is gives me a reason to have a look at it. This is quite a nice way to override configurations.

                Comment


                • #9
                  I just hacked together a proof of concept implementation that stores all key/value pairs of a PropertyMap field of a NodeEntity to the underlying neo4j node. PropertyMap is an newly introduced PropertyContainer like interface (see below). A own FieldAccessorFactory handles getting and setting this field.
                  Sounds good.

                  Some kind of FieldAccessorFactory registry would be great. I now ended up configuring the whole Spring Data Graph stuff in XML bean notation, because the java config of course does not know anything about my extended NodeDelegatingFieldAccessorFactory and it has to be wired in manually. May there be simpler alternatives to just "replace" the NodeDelegatingFieldAccessorFactory bean with an own implementation?
                  Right. You shouldn't need to configure everything in XML just override the definition of "nodeEntityStateFactory" with an equivalent of this bean definition:

                  PHP Code:
                      @Bean
                      
                  public NodeEntityStateFactory nodeEntityStateFactory() throws Exception {
                          
                  NodeEntityStateFactory entityStateFactory = new NodeEntityStateFactory();
                          
                  entityStateFactory.setGraphDatabaseContext(graphDatabaseContext());
                          
                  entityStateFactory.setEntityManagerFactory(entityManagerFactory);
                          final 
                  NodeDelegatingFieldAccessorFactory nodeDelegatingFieldAccessorFactory = new NodeDelegatingFieldAccessorFactory(graphDatabaseContext());
                          
                  entityStateFactory.setNodeDelegatingFieldAccessorFactory(nodeDelegatingFieldAccessorFactory);
                          return 
                  entityStateFactory;
                      } 
                  i.e. something like this should be enough
                  PHP Code:
                     <bean id="nodeEntityStateFactory" class="NodeEntityStateFactory">
                        <
                  property name="graphDatabaseContext" ref="graphDatabaseContext"/>
                        <
                  property name="entityManagerFactory" ref="entityManagerFactory"/>
                        <
                  property name="nodeDelegatingFieldAccessorFactory">
                            <
                  bean class="MyNodeDelegatingFieldAccessorFactory">
                                 <
                  constructor-arg ref="graphDatabaseContext"/>
                            </
                  bean>
                        </
                  property>
                     </
                  bean

                  The problem with the inlined map is that this map does not know which properties belong to the map and which belong to the entity itself, when it is loaded.
                  I'd solve that by prefixing the properties with "fieldName." just as you proposed in your initial post. So you can easily filter the properties that belong to your map by iterating over node.getPropertyNames() and filtering for the field.getName() that you can get from your FieldAccessorFactory.

                  I introduced an interface PropertyMap that is the same as PropertyContainer, but without the getGraphDatabase() method. An implementation of this PropertyMap uses a Map<String, Object> as a delegate to actually store the key/value pairs and is used in the DynamicPropertiesFieldAccessor that handles properties of type PropertyMap.

                  My next step will be to try to store the dynamic map in a separate node so that I do not have to know which keys are already handled by other FieldAccessors on the entity itself.
                  Looking forward to that and to merge it into SDG itself.

                  Thanks a lot for your efforts.

                  Michael

                  Comment


                  • #10
                    I'm on vacation for the next two weeks. I did not manage to prepare this stuff to be merged into SDG before my holiday. Storing of inline properties works so far. The next step will be to store properties on a separate node. I had no time to work on this yet.
                    So expect it to be ready in around three weeks.

                    Comment


                    • #11
                      Ok, that fits perfectly, I'm also off for 3 weeks.

                      Enjoy your vacation.

                      Michael

                      Comment


                      • #12
                        Thanks, I will. Enjoy yours as well.

                        James

                        Comment


                        • #13
                          Please see https://github.com/SpringSource/spri...a-graph/pull/8 for a first implementation of DynamicProperties.
                          Currently, only the "inline" properties are implemented. I'd be glad if you could have a look and tell me what you think.

                          What bugs me at the moment is the fact, that these dynamic properties cannot be used with detached entities, although it would be feasible. Can you see a way to do that?

                          Comment


                          • #14
                            Will look into it and merge it, what is the actual issue when changing map properties on a detached entity?

                            Michael

                            Comment


                            • #15
                              I did not find a way to have a ManagedPrefixedDynamicProperties container for the NodeEntities ready also in detached mode. It currently works exactly like in the AbstractNodeRelationshipFieldAccessor, where a ManagedFieldAccessorSet is created in the getValue method of the accessor.
                              It's currently not possible to do something like:
                              PHP Code:
                              @NodeEntity
                              public class Person {
                                  
                              DynamicProperties props;
                              }

                              Person p = new Person();
                              p.props.setProperty("foo""bar"); 
                              ...because the props field is null.

                              Bus as I mentioned in the comment of the pull request: We can live with that. :-)

                              Comment

                              Working...
                              X