Announcement Announcement Module
Collapse
No announcement yet.
wsdl generation not thread safe? Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • wsdl generation not thread safe?

    We're experiencing failures with wsdl generation when we have multiple concurrent clients. The clients are Flex applications that request the wsdl every second, then invoke a web service. Here is the failure:

    Code:
    2007-09-04 11:31:06,203 DEBUG [com.ssn.ods.webservice.SsnMessageDispatcherServlet] doService
    2007-09-04 11:31:06,203 DEBUG [com.ssn.ods.webservice.SsnMessageDispatcherServlet] Could not complete request
    java.lang.NullPointerException
    	at com.sun.org.apache.xerces.internal.dom.ParentNode.nodeListItem(ParentNode.java:814)
    	at com.sun.org.apache.xerces.internal.dom.ParentNode.item(ParentNode.java:828)
    	at com.ibm.wsdl.util.xml.DOM2Writer.print(Unknown Source)
    	at com.ibm.wsdl.util.xml.DOM2Writer.print(Unknown Source)
    	at com.ibm.wsdl.util.xml.DOM2Writer.print(Unknown Source)
    	at com.ibm.wsdl.util.xml.DOM2Writer.print(Unknown Source)
    	at com.ibm.wsdl.util.xml.DOM2Writer.serializeAsXML(Unknown Source)
    	at com.ibm.wsdl.extensions.schema.SchemaSerializer.marshall(Unknown Source)
    	at com.ibm.wsdl.xml.WSDLWriterImpl.printExtensibilityElements(Unknown Source)
    	at com.ibm.wsdl.xml.WSDLWriterImpl.printTypes(Unknown Source)
    	at com.ibm.wsdl.xml.WSDLWriterImpl.printDefinition(Unknown Source)
    	at com.ibm.wsdl.xml.WSDLWriterImpl.writeWSDL(Unknown Source)
    	at com.ibm.wsdl.xml.WSDLWriterImpl.getDocument(Unknown Source)
    	at org.springframework.ws.wsdl.wsdl11.Wsdl4jDefinition.getSource(Wsdl4jDefinition.java:75)
    	at org.springframework.ws.wsdl.wsdl11.DynamicWsdl11Definition.getSource(DynamicWsdl11Definition.java:82)
    	at org.springframework.ws.transport.http.WsdlDefinitionHandlerAdapter.handle(WsdlDefinitionHandlerAdapter.java:125)
    	at org.springframework.ws.transport.http.MessageDispatcherServlet.doService(MessageDispatcherServlet.java:155)
    	at com.ssn.ods.webservice.SsnMessageDispatcherServlet.doService(SsnMessageDispatcherServlet.java:41)
    	at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:475)
    	at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:430)
    	at javax.servlet.http.HttpServlet.service(HttpServlet.java:690)
    	at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
    	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:269)
    	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
    	at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:265)
    	at org.acegisecurity.intercept.web.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:107)
    	at org.acegisecurity.intercept.web.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:72)
    	at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:275)
    	at org.acegisecurity.ui.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:110)
    	at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:275)
    	at org.acegisecurity.wrapper.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:81)
    	at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:275)
    	at org.acegisecurity.ui.basicauth.BasicProcessingFilter.doFilter(BasicProcessingFilter.java:175)
    	at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:275)
    	at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:286)
    	at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:275)
    	at org.acegisecurity.util.FilterChainProxy.doFilter(FilterChainProxy.java:149)
    	at org.acegisecurity.util.FilterToBeanProxy.doFilter(FilterToBeanProxy.java:98)
    	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
    	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
    	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:210)
    	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:174)
    	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
    	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117)
    	at org.apache.catalina.valves.FastCommonAccessLogValve.invoke(FastCommonAccessLogValve.java:482)
    	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108)
    	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:151)
    	at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:870)
    	at org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:665)
    	at org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:528)
    	at org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:81)
    	at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:685)
    	at java.lang.Thread.run(Thread.java:595)
    Everything works for a few iterations, sometimes as many as 250, then we get the error above. We are using the DynamicWsdl11Definition. Here is a snippet from the Spring web service context:

    Code:
    <bean id="OutageDetection" class="org.springframework.ws.wsdl.wsdl11.DynamicWsdl11Definition">
            <description>
                Dynamically builds a WSDL from the xsd.
                Try: http://localhost:55555/ods/webservice/NAME.wsdl
            </description>
            <property name="builder">
                <bean class="org.springframework.ws.wsdl.wsdl11.builder.XsdBasedSoap11Wsdl4jDefinitionBuilder">
                    <property name="schema" value="schema.message/OutageDetection.xsd"/>
                    <property name="portTypeName" value="OutageDetectionPort"/>
                    <property name="locationUri" value="http://${host.name}:${tomcat.http.port}/ods/webservice"/>
                    <property name="targetNamespace" value="urn:com:ssn:ods:webservice:OutageDetection"/>
                </bean>
            </property>
        </bean>
    Has anyone run into this? Should we expect the wsdl generation to support multiple concurrent clients? I'm able to reproduce this, and my test does not execute the web service. It just retrieves the wsdl with a URL like this:

    http://localhost:50424/ods/webservic...Detection.wsdl

  • #2
    That's weird, because the WSDL should be generated only once, and then cached. After that, it is simply written to the HTTP response every time. So to me it seems like this is a WSDL4J problem.

    Comment


    • #3
      We are experience the exact same problem. We are using Jakarta JMeter to stress test our system, and in our last run we got 2 errors out of 850 requests. Does there exist a fix for this problem?

      Rune Hamnvik

      Comment


      • #4
        THere is no fix yet, but if you create a JIRA issue, I will take a look at it for 1.0.2.

        Comment


        • #5
          I was not able to find a fix. We capture the wsdl and then serve it up statically. It is unfortunate, as it adds a manual step when the schema for the web service changes.

          Comment


          • #6
            JIRA issue SWS-200 created.

            Rune Hamnvik

            Comment


            • #7
              ... and fixed. As it turns out, WSDL4J (the library we use) is not thread-safe, so I put a synchronized block around it. Now it works.

              Comment


              • #8
                Thanks for quick response. Impressive!
                Do you know when 1.0.1 will be released?

                Rune Hamnvik

                Comment


                • #9
                  By the end of this week, so tomorrow, or Saturday.

                  Comment


                  • #10
                    I've just upgraded our installation to Spring WS 1.0.1 and the problem seems to be worse. Before I was able to execute many threads concurrently for as many as 60 iterations before running into the problem. Now it seems to happen in the first set of iterations. More specifically, the test has 7 threads that each try to get the WSDL 100 times. The 6th thread now fails in its first attempt to get the WSDL. The message follows. You'll see a class called SsnMessageDispatcherServlet. It does nothing but wrap MessageDispatcherServlet. Interestingly, the first 5 threads sucessfully execute all 100 attempts.

                    Code:
                    java.lang.IllegalArgumentException: type: -1
                    	at org.apache.xerces.dom.DeferredDocumentImpl.getNodeObject(Unknown Source)
                    	at org.apache.xerces.dom.DeferredDocumentImpl.synchronizeChildren(Unknown Source)
                    	at org.apache.xerces.dom.ParentNode.getFirstChild(Unknown Source)
                    	at org.apache.xml.serializer.TreeWalker.traverse(TreeWalker.java:147)
                    	at org.apache.xalan.transformer.TransformerIdentityImpl.transform(TransformerIdentityImpl.java:389)
                    	at org.springframework.ws.transport.http.WsdlDefinitionHandlerAdapter.handle(WsdlDefinitionHandlerAdapter.java:128)
                    	at org.springframework.ws.transport.http.MessageDispatcherServlet.doService(MessageDispatcherServlet.java:155)
                    	at com.ssn.ods.webservice.SsnMessageDispatcherServlet.doService(SsnMessageDispatcherServlet.java:41)
                    	at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:475)
                    	at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:430)
                    	at javax.servlet.http.HttpServlet.service(HttpServlet.java:690)
                    	at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
                    	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:269)
                    	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
                    	at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:265)
                    	at org.acegisecurity.intercept.web.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:107)
                    	at org.acegisecurity.intercept.web.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:72)
                    	at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:275)
                    	at org.acegisecurity.ui.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:110)
                    	at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:275)
                    	at org.acegisecurity.wrapper.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:81)
                    	at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:275)
                    	at org.acegisecurity.ui.basicauth.BasicProcessingFilter.doFilter(BasicProcessingFilter.java:175)
                    	at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:275)
                    	at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:286)
                    	at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:275)
                    	at org.acegisecurity.util.FilterChainProxy.doFilter(FilterChainProxy.java:149)
                    	at org.acegisecurity.util.FilterToBeanProxy.doFilter(FilterToBeanProxy.java:98)
                    	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
                    	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
                    	at com.ssn.ods.container.JpaEntityManagerInjectorFilter.doFilter(JpaEntityManagerInjectorFilter.java:45)
                    	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
                    	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
                    	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:210)
                    	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:174)
                    	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
                    	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117)
                    	at org.apache.catalina.valves.FastCommonAccessLogValve.invoke(FastCommonAccessLogValve.java:482)
                    	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108)
                    	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:151)
                    	at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:870)
                    	at org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:665)
                    	at org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:528)
                    	at org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:81)
                    	at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:685)
                    	at java.lang.Thread.run(Thread.java:595)
                    2007-10-01 14:11:02,974 DEBUG [com.ssn.ods.webservice.SsnMessageDispatcherServlet] Could not complete request
                    java.lang.NullPointerException
                    	at org.apache.xml.serializer.TreeWalker.dispatachChars(TreeWalker.java:246)
                    	at org.apache.xml.serializer.TreeWalker.startNode(TreeWalker.java:416)
                    	at org.apache.xml.serializer.TreeWalker.traverse(TreeWalker.java:145)
                    	at org.apache.xalan.transformer.TransformerIdentityImpl.transform(TransformerIdentityImpl.java:389)
                    	at org.springframework.ws.transport.http.WsdlDefinitionHandlerAdapter.handle(WsdlDefinitionHandlerAdapter.java:128)
                    	at org.springframework.ws.transport.http.MessageDispatcherServlet.doService(MessageDispatcherServlet.java:155)
                    	at com.ssn.ods.webservice.SsnMessageDispatcherServlet.doService(SsnMessageDispatcherServlet.java:41)
                    	at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:475)
                    	at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:430)
                    	at javax.servlet.http.HttpServlet.service(HttpServlet.java:690)
                    	at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
                    	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:269)
                    	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
                    	at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:265)
                    	at org.acegisecurity.intercept.web.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:107)
                    	at org.acegisecurity.intercept.web.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:72)
                    	at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:275)
                    	at org.acegisecurity.ui.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:110)
                    	at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:275)
                    	at org.acegisecurity.wrapper.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:81)
                    	at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:275)
                    	at org.acegisecurity.ui.basicauth.BasicProcessingFilter.doFilter(BasicProcessingFilter.java:175)
                    	at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:275)
                    	at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:286)
                    	at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:275)
                    	at org.acegisecurity.util.FilterChainProxy.doFilter(FilterChainProxy.java:149)
                    	at org.acegisecurity.util.FilterToBeanProxy.doFilter(FilterToBeanProxy.java:98)
                    	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
                    	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
                    	at com.ssn.ods.container.JpaEntityManagerInjectorFilter.doFilter(JpaEntityManagerInjectorFilter.java:45)
                    	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
                    	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
                    	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:210)
                    	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:174)
                    	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
                    	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117)
                    	at org.apache.catalina.valves.FastCommonAccessLogValve.invoke(FastCommonAccessLogValve.java:482)
                    	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108)
                    	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:151)
                    	at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:870)
                    	at org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:665)
                    	at org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:528)
                    	at org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:81)
                    	at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:685)
                    	at java.lang.Thread.run(Thread.java:595)
                    2007-10-01 14:11:02,974 DEBUG [com.ssn.ods.container.JpaEntityManagerInjectorFilter] comitting transaction
                    2007-10-01 14:11:03,021 DEBUG [com.ssn.ods.webservice.SsnMessageDispatcherServlet] Could not complete request
                    org.w3c.dom.DOMException: NAMESPACE_ERR: An attempt is made to create or change an object in a way which is incorrect with regard to namespaces.
                    	at org.apache.xerces.dom.ElementNSImpl.setName(Unknown Source)
                    	at org.apache.xerces.dom.ElementNSImpl.<init>(Unknown Source)
                    	at org.apache.xerces.dom.CoreDocumentImpl.createElementNS

                    Comment


                    • #11
                      Damn, I fixed it for Jetty, but didn't check Tomcat.

                      I've reopened the issue, and will fix it for 1.0.2. Hopefully, for good this time.

                      Comment


                      • #12
                        I cannot reproduce this with either Tomcat 6.0.14 and 5.5.23. I used Apache Bench (ab) to request the WDSL of the echo sample app a 1000 times, using 100 concurrent threads. From that test, I get the following:

                        Code:
                        >ab -n 1000 -c 100 http://192.168.0.2:8080/echo/echo.wsdl
                        This is ApacheBench, Version 1.3d <$Revision: 1.73 $> apache-1.3
                        Copyright (c) 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
                        Copyright (c) 1998-2002 The Apache Software Foundation, http://www.apache.org/
                        
                        Benchmarking 192.168.0.2 (be patient)
                        Completed 100 requests
                        Completed 200 requests
                        Completed 300 requests
                        Completed 400 requests
                        Completed 500 requests
                        Completed 600 requests
                        Completed 700 requests
                        Completed 800 requests
                        Completed 900 requests
                        Finished 1000 requests
                        Server Software:        Apache-Coyote/1.1                                  
                        Server Hostname:        192.168.0.2
                        Server Port:            8080
                        
                        Document Path:          /echo/echo.wsdl
                        Document Length:        2060 bytes
                        
                        Concurrency Level:      100
                        Time taken for tests:   2.647 seconds
                        Complete requests:      1000
                        Failed requests:        0
                        Broken pipe errors:     0
                        Total transferred:      2188186 bytes
                        HTML transferred:       2062060 bytes
                        Requests per second:    377.79 [#/sec] (mean)
                        Time per request:       264.70 [ms] (mean)
                        Time per request:       2.65 [ms] (mean, across all concurrent requests)
                        Transfer rate:          826.67 [Kbytes/sec] received

                        Comment


                        • #13
                          Thanks Arjen,

                          I'll give it another try and double check my tests. If they still fail, I'll give it a try with ApaceBench.

                          Ken

                          Comment


                          • #14
                            Everything looks good. The last test failure was my fault. I hadn't cleaned everything up before testing the update.

                            Thank you very much for looking into this, and for resolving the problem so quickly.

                            Comment


                            • #15
                              Solution..

                              Hi,


                              I had hit the same issue.

                              The exact scenario to reproduce it is: Run a JMeter HTTP GET request test (NOT webservices test) for the wsdl url with thread size (users)=10; ramp-up period=0; loop-count=50

                              As already mentioned in this thread, when there are too many concurrent wsdl file requests, the spring app will die since the operations performed during wsdl file generation are not thread-safe.

                              The solution is to either cache the wsdl in the client end or better, to do away with the dynamic wsdl generation. The latter one is better if you consider the fact that anyone in your nw could use a jmeter test to down your services :-D

                              I would dynamically generate the wsdl file for code or environment change, and serve this file from the services. The spring xml config would change like:

                              Code:
                              <bean id="myservice" class="org.springframework.ws.wsdl.wsdl11.SimpleWsdl11Definition">
                                      <constructor-arg value="/WEB-INF/myservice.wsdl.wsdl"/>
                              </bean>
                              Hope that helps..

                              Comment

                              Working...
                              X