Announcement Announcement Module
Collapse
No announcement yet.
Neo4j rest painfully slow on Heroku Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • Neo4j rest painfully slow on Heroku

    I have deployed my app on Heroku as per the instructions here but although the app works, it is painfully slow.

    The app runs as a restful web service and communicates with neo4j rest. When running the web service and neo4j on my local machine the latency between submitting the url and receiving a response is under 250ms, with around 240ms consumed by neo4j rest.

    However, when I deploy to heroku, the total latency averages 6,800ms with approximately 5,500ms consumed by neo4j. Is this a heroku problem or do I have a problem with my setup?

    Any help would be greatly appreciated!

    The controller
    Code:
    @Controller
    @RequestMapping("/")
    public class RestController {
    	
        @Autowired
        private UserRepository userRep;
        
        @RequestMapping(value="/TIMING", method = RequestMethod.GET)
        public @ResponseBody ModelMap timing( @RequestParam("id") String id, ModelMap map) {
        	
        	long bar = System.currentTimeMillis();
        	
        	User user = userRep.findOne(Long.parseLong(id));
    
        	long foo = System.currentTimeMillis();
        	 
        	map.addAttribute("userId", user.getGraphId());
        	
        	return map.addAttribute("timing", foo - bar);
        }
    web.xml
    Code:
    <web-app id="WebApp_ID" version="2.4"
    	xmlns="http://java.sun.com/xml/ns/j2ee" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    	xsi:schemaLocation="http://java.sun.com/xml/ns/j2ee 
    	http://java.sun.com/xml/ns/j2ee/web-app_2_4.xsd">
    
    	<display-name>App</display-name>
    
    	<servlet>
    		<servlet-name>mvc-dispatcher</servlet-name>
    		<servlet-class>org.springframework.web.servlet.DispatcherServlet</servlet-class>
    		<load-on-startup>1</load-on-startup>
    	</servlet>
    
    	<servlet-mapping>
    		<servlet-name>mvc-dispatcher</servlet-name>
    		<url-pattern>/</url-pattern>
    	</servlet-mapping>
    
    	<context-param>
    		<param-name>contextConfigLocation</param-name>
            <param-value>
    			/WEB-INF/applicationContext.xml
    		</param-value>
    	</context-param>
     
    	<listener>
    		<listener-class>org.springframework.web.context.ContextLoaderListener</listener-class>
    	</listener>
    </web-app>
    applicationContext.xml
    Code:
    <?xml version="1.0" encoding="UTF-8" standalone="no"?>
    <beans xmlns="http://www.springframework.org/schema/beans"
           xmlns:context="http://www.springframework.org/schema/context"
           xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
           xmlns:neo4j="http://www.springframework.org/schema/data/neo4j"
           xmlns:tx="http://www.springframework.org/schema/tx"
           xsi:schemaLocation="
           	http://www.springframework.org/schema/beans 
           	http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
    		http://www.springframework.org/schema/context 
    		http://www.springframework.org/schema/context/spring-context-3.0.xsd
    		http://www.springframework.org/schema/data/neo4j
    		http://www.springframework.org/schema/data/neo4j/spring-neo4j-2.0.xsd
    		http://www.springframework.org/schema/tx
    		http://www.springframework.org/schema/tx/spring-tx-2.5.xsd">
    
        <context:annotation-config/>
        <context:component-scan base-package="com.getshout.app">
            <context:exclude-filter type="annotation" expression="org.springframework.stereotype.Controller"/>
        </context:component-scan>
    
    	<context:spring-configured/>
    
    	<neo4j:config graphDatabaseService="graphDatabaseService"/>
    	<bean id="graphDatabaseService" 
    		class="org.springframework.data.neo4j.rest.SpringRestGraphDatabase">
    
    
        	<constructor-arg index="0" value="http://123456.hosted.neo4j.org:7981/db/data/" />
        	<constructor-arg index="1" value="abc123456" />
        	<constructor-arg index="2" value="abd123456" />
        	 
    	</bean>
    
        <neo4j:repositories base-package="com.getshout.app"/>
        <tx:annotation-driven mode="proxy"/>
    </beans>
    mvc-dispatcher-servlet.xml
    Code:
    <beans xmlns="http://www.springframework.org/schema/beans"
    	xmlns:context="http://www.springframework.org/schema/context"
    	xmlns:mvc="http://www.springframework.org/schema/mvc" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    	xsi:schemaLocation="
            http://www.springframework.org/schema/beans     
            http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
            http://www.springframework.org/schema/context 
            http://www.springframework.org/schema/context/spring-context-3.0.xsd
            http://www.springframework.org/schema/mvc
            http://www.springframework.org/schema/mvc/spring-mvc-3.0.xsd">
    
    
    	<context:component-scan base-package="com.getshout.app.controller" />
    	<mvc:annotation-driven />
    
    	<bean
    		class="org.springframework.web.servlet.view.InternalResourceViewResolver">
    		<property name="order" value="1" /> 
    		<property name="prefix">
    			<value>/WEB-INF/pages/</value>
    		</property>
    		<property name="suffix">
    			<value>.jsp</value>
    		</property>
    	</bean>
    </beans>
    Last edited by MichaelJaniak; Aug 16th, 2012, 03:45 PM.

  • #2
    Hi Michael,

    Can you get in touch directly to discuss this? I can take a look at your instance to see if it's behaving or not.

    Sincerely,

    Julian.




    Originally posted by MichaelJaniak View Post
    I have deployed my app on Heroku as per the instructions here but although the app works, it is painfully slow.

    The app runs as a restful web service and communicates with neo4j rest. When running the web service and neo4j on my local machine the latency between submitting the url and receiving a response is under 250ms, with around 240ms consumed by neo4j rest.

    However, when I deploy to heroku, the total latency averages 6,800ms with approximately 5,500ms consumed by neo4j. Is this a heroku problem or do I have a problem with my setup?

    Any help would be greatly appreciated!

    The controller
    Code:
    @Controller
    @RequestMapping("/")
    public class RestController {
    	
        @Autowired
        private UserRepository userRep;
        
        @RequestMapping(value="/TIMING", method = RequestMethod.GET)
        public @ResponseBody ModelMap timing( @RequestParam("id") String id, ModelMap map) {
        	
        	long bar = System.currentTimeMillis();
        	
        	User user = userRep.findOne(Long.parseLong(id));
    
        	long foo = System.currentTimeMillis();
        	 
        	map.addAttribute("userId", user.getGraphId());
        	
        	return map.addAttribute("timing", foo - bar);
        }
    web.xml
    Code:
    <web-app id="WebApp_ID" version="2.4"
    	xmlns="http://java.sun.com/xml/ns/j2ee" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    	xsi:schemaLocation="http://java.sun.com/xml/ns/j2ee 
    	http://java.sun.com/xml/ns/j2ee/web-app_2_4.xsd">
    
    	<display-name>App</display-name>
    
    	<servlet>
    		<servlet-name>mvc-dispatcher</servlet-name>
    		<servlet-class>org.springframework.web.servlet.DispatcherServlet</servlet-class>
    		<load-on-startup>1</load-on-startup>
    	</servlet>
    
    	<servlet-mapping>
    		<servlet-name>mvc-dispatcher</servlet-name>
    		<url-pattern>/</url-pattern>
    	</servlet-mapping>
    
    	<context-param>
    		<param-name>contextConfigLocation</param-name>
            <param-value>
    			/WEB-INF/applicationContext.xml
    		</param-value>
    	</context-param>
     
    	<listener>
    		<listener-class>org.springframework.web.context.ContextLoaderListener</listener-class>
    	</listener>
    </web-app>
    applicationContext.xml
    Code:
    <?xml version="1.0" encoding="UTF-8" standalone="no"?>
    <beans xmlns="http://www.springframework.org/schema/beans"
           xmlns:context="http://www.springframework.org/schema/context"
           xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
           xmlns:neo4j="http://www.springframework.org/schema/data/neo4j"
           xmlns:tx="http://www.springframework.org/schema/tx"
           xsi:schemaLocation="
           	http://www.springframework.org/schema/beans 
           	http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
    		http://www.springframework.org/schema/context 
    		http://www.springframework.org/schema/context/spring-context-3.0.xsd
    		http://www.springframework.org/schema/data/neo4j
    		http://www.springframework.org/schema/data/neo4j/spring-neo4j-2.0.xsd
    		http://www.springframework.org/schema/tx
    		http://www.springframework.org/schema/tx/spring-tx-2.5.xsd">
    
        <context:annotation-config/>
        <context:component-scan base-package="com.getshout.app">
            <context:exclude-filter type="annotation" expression="org.springframework.stereotype.Controller"/>
        </context:component-scan>
    
    	<context:spring-configured/>
    
    	<neo4j:config graphDatabaseService="graphDatabaseService"/>
    	<bean id="graphDatabaseService" 
    		class="org.springframework.data.neo4j.rest.SpringRestGraphDatabase">
    
    
        	<constructor-arg index="0" value="http://123456.hosted.neo4j.org:7981/db/data/" />
        	<constructor-arg index="1" value="abc123456" />
        	<constructor-arg index="2" value="abd123456" />
        	 
    	</bean>
    
        <neo4j:repositories base-package="com.getshout.app"/>
        <tx:annotation-driven mode="proxy"/>
    </beans>
    mvc-dispatcher-servlet.xml
    Code:
    <beans xmlns="http://www.springframework.org/schema/beans"
    	xmlns:context="http://www.springframework.org/schema/context"
    	xmlns:mvc="http://www.springframework.org/schema/mvc" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    	xsi:schemaLocation="
            http://www.springframework.org/schema/beans     
            http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
            http://www.springframework.org/schema/context 
            http://www.springframework.org/schema/context/spring-context-3.0.xsd
            http://www.springframework.org/schema/mvc
            http://www.springframework.org/schema/mvc/spring-mvc-3.0.xsd">
    
    
    	<context:component-scan base-package="com.getshout.app.controller" />
    	<mvc:annotation-driven />
    
    	<bean
    		class="org.springframework.web.servlet.view.InternalResourceViewResolver">
    		<property name="order" value="1" /> 
    		<property name="prefix">
    			<value>/WEB-INF/pages/</value>
    		</property>
    		<property name="suffix">
    			<value>.jsp</value>
    		</property>
    	</bean>
    </beans>

    Comment


    • #3
      Please note that the fine grained operations of SDN are right now not working very well over higher-latency connections.
      I recommend more cypher queries and traversals that directly run on the sever over the attribute based approach that uses individual REST calls.

      If you are just retrieving single values it is much faster to use cypher to do that:

      Code:
      start n=node({id}) return n.graphId
      and provide "id" as parameter.

      You can use the query Method of the SpringRestGraphDatabase.

      Michael

      Comment


      • #4
        That's amazing, solved the problem. Who would have thought.

        Thank you Michael!

        Comment


        • #5
          Can you explain a bit further. I have the exact same issue. I have Cypher queries in my repo, but it takes forever on Heroku.

          Thanks

          Mark

          Comment


          • #6
            The first thing that helped was as per Michael's suggestion, I only use cypher queries in the repo. I don't use any SDN query methods because they're all slow on Heroku. That alone got me down from 15seconds on some queries to 7 seconds.

            After that I removed all @Fetch annotations from my @RelationshipEntity objects (and @NodeEntity if you've got them there) and that got me down to 1 second.

            Finally, I merged all my look ups into one big cypher query returning multiple columns, and that got me sub 500ms, even down to 200ms, obviously depending on external factors but you get the idea.

            Comment


            • #7
              Thanks for the description.

              Yeah, unfortunately, Some of my queries couldn't be solved with Cypher and I had to do things the long way. Or in some cases the Cypher version wouldn't work the same as the SDN repo given queries. Probably my fault.

              Mark

              Comment

              Working...
              X