Announcement Announcement Module
Collapse
No announcement yet.
JMSTemplate sending thousnads of messages Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • JMSTemplate sending thousnads of messages

    I have a standalone application and a test class that sends "faked" messages to the 'in bound queue' which my application will listen on.

    Problem is I want to mimic sending 5 - 10,000 messages to the queue.

    I have put the jmsTemplate.send() in a for loop with a counter.
    Works great with 1 - 1000 messages, but when I jack it up to 5000 or more I get an error.


    Is there a better way to do this so I can get a lot of messages on the queue?
    If I am having trouble sending 5000 messages will I also have problems in my actual application listening and then routing these messages?

    Any guidance would be appreciated!


    Code:
        @Test
        public void generateOneThousandGoodMessagesOnInboundQueue() throws  Exception{
            String msg  = fileProcessingUtil.readMessageAsString("obtain_sample_message);
            for(int i = 1; i <= 5000; i++){
                    sendMessage(msg);
    
            }
    
          }
    
    
         private void sendMessage(final String messageToSend){
            jmsTemplate.send(new MessageCreator() {
                public Message createMessage(Session session) throws JMSException {
                    return session.createTextMessage( messageToSend );
                }
            } );
    
          }

    Code:
    org.springframework.jms.UncategorizedJmsException: Uncategorized exception occured during JMS processing; nested exception is javax.jms.JMSException: MQJMS2005: failed to create MQQueueManager for 'fheg-efmq.fheg.follett.com:INTSVR'; nested exception is com.ibm.mq.MQException: MQJE001: An MQException occurred: Completion Code 2, Reason 2059
    MQJE011: Socket connection attempt refused
    	at org.springframework.jms.support.JmsUtils.convertJmsAccessException(JmsUtils.java:316)
    	at org.springframework.jms.support.JmsAccessor.convertJmsAccessException(JmsAccessor.java:168)
    	at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:469)
    	at org.springframework.jms.core.JmsTemplate.send(JmsTemplate.java:543)
    	at org.springframework.jms.core.JmsTemplate.send(JmsTemplate.java:529)
    	at com.follett.fheg.integration.MessageIntegrationTest.sendMessage(MessageIntegrationTest.java:110)
    	at com.follett.fheg.integration.MessageIntegrationTest.generateOneThousandGoodMessagesOnInboundQueue(MessageIntegrationTest.java:93)
    	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
    	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
    	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
    	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
    	at org.springframework.test.context.junit4.statements.RunBeforeTestMethodCallbacks.evaluate(RunBeforeTestMethodCallbacks.java:74)
    	at org.springframework.test.context.junit4.statements.RunAfterTestMethodCallbacks.evaluate(RunAfterTestMethodCallbacks.java:82)
    	at org.springframework.test.context.junit4.statements.SpringRepeat.evaluate(SpringRepeat.java:72)
    	at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:240)
    	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:48)
    	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:242)
    	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:58)
    	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:240)
    	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:48)
    	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:233)
    	at org.springframework.test.context.junit4.statements.RunBeforeTestClassCallbacks.evaluate(RunBeforeTestClassCallbacks.java:61)
    	at org.springframework.test.context.junit4.statements.RunAfterTestClassCallbacks.evaluate(RunAfterTestClassCallbacks.java:70)
    	at org.junit.runners.ParentRunner.run(ParentRunner.java:303)
    	at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.run(SpringJUnit4ClassRunner.java:180)
    	at org.junit.runner.JUnitCore.run(JUnitCore.java:157)
    	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:97)
    	at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:192)
    	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:60)
    	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    	at com.intellij.rt.execution.application.AppMain.main(AppMain.java:115)
    Caused by: javax.jms.JMSException: MQJMS2005: failed to create MQQueueManager for 'fheg-efmq.fheg.follett.com:INTSVR'
    	at com.ibm.mq.jms.services.ConfigEnvironment.newException(ConfigEnvironment.java:586)
    	at com.ibm.mq.jms.MQConnection.createQM(MQConnection.java:2301)
    	at com.ibm.mq.jms.MQConnection.createQMNonXA(MQConnection.java:1703)
    	at com.ibm.mq.jms.MQQueueConnection.<init>(MQQueueConnection.java:157)
    	at com.ibm.mq.jms.MQQueueConnectionFactory.createQueueConnection(MQQueueConnectionFactory.java:185)
    	at com.ibm.mq.jms.MQQueueConnectionFactory.createQueueConnection(MQQueueConnectionFactory.java:112)
    	at com.ibm.mq.jms.MQQueueConnectionFactory.createConnection(MQQueueConnectionFactory.java:1050)
    	at org.springframework.jms.support.JmsAccessor.createConnection(JmsAccessor.java:184)
    	at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:456)
    	... 35 more

  • #2
    Can you post your JmsTemplate and ConnectionFactory configuration ?

    Comment


    • #3
      Sure thing! Thanks for responding.

      I was able to figure something out and seems to be working so far. I was able to get 8,000 messages sent to my in bound queue via a testing class.
      Then used the actual application to read (listen) and route those messages to the appropriate destination queue.

      Problem is I am not sure exactly why it works.

      What I did was add a Spring CachingConnectionFactory to wrap the MQConnectionFactory. Then in the MQConnectionFactory I added a property 'useConnectionPooling' and set this to true.



      Code:
      <?xml version="1.0" encoding="UTF-8"?>
      <beans xmlns="http://www.springframework.org/schema/beans"
             xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
             xmlns:context="http://www.springframework.org/schema/context"
             xmlns:tx="http://www.springframework.org/schema/tx"
             xmlns:aop="http://www.springframework.org/schema/aop"
      
             xsi:schemaLocation="http://www.springframework.org/schema/beans
             						http://www.springframework.org/schem...-beans-3.0.xsd
             						http://www.springframework.org/schema/context
             						http://www.springframework.org/schem...ontext-3.0.xsd
             						http://www.springframework.org/schema/aop
             						http://www.springframework.org/schem...spring-aop.xsd
             						http://www.springframework.org/schema/tx
             						http://www.springframework.org/schema/tx/spring-tx.xsd">
      
      
      
          <context:property-placeholder location="file:///etc///env///OrderRouter///mms_message_router///mq.properties, file:///etc///env///OrderRouter///mms_message_router///mailserver.properties,file:///etc///env///OrderRouter///mms_message_router///application.properties"/>
      
      
          <tx:advice id="txAdvice" transaction-manager="txManager">
             <tx:attributes>
                 <tx:method name="*" timeout="30"/>
             </tx:attributes>
          </tx:advice>
      
      
          <!-- if any of the methods in the service package throw a RuntimeException or (ERROR) rollback the transaction -->
      
          <aop:config>
              <aop:pointcut id="messageRoutingServiceOperation" expression="execution(* com.my.company.service.*.*(..))"/>
              <aop:advisor advice-ref="txAdvice" pointcut-ref="messageRoutingServiceOperation"/>
          </aop:config>
      
          <!-- Configure the queues (JmsTemplates) necessary for routing orders -->
      
      
          <bean id="abstractJmsTemplate" class="org.springframework.jms.core.JmsTemplate" abstract="true">
            <property name="connectionFactory" ref="cachedConnectionFactory"/>
            <property name="sessionTransacted" value="true"/>
          </bean>
      
          <bean id="jmsTemplateOrdersNonPilot" class="org.springframework.jms.core.JmsTemplate" parent="abstractJmsTemplate">
            <property name="defaultDestinationName" value="queue:///${mq.mmsOrdersOutboundNonPilot}?targetClient=1" />
         </bean>
      
          <bean id="jmsTemplateOrdersPilot" class="org.springframework.jms.core.JmsTemplate" parent="abstractJmsTemplate">
            <property name="defaultDestinationName" value="queue:///${mq.mmsOrdersOutboundPilot}?targetClient=1" />
         </bean>
      
          <bean id="deadLetterQ" class="org.springframework.jms.core.JmsTemplate" parent="abstractJmsTemplate">
            <property name="defaultDestinationName" value="queue:///${mq.deadLetterQueue}?targetClient=1" />
         </bean>
      
          <!-- ***  -->
      
      
          <bean id="jmsTemplateFactory" class="com.my.company.messagebuilder.JmsQueueMessageBuilder">
              <property name="jmsTemplateOrdersNonPilot" ref="jmsTemplateOrdersNonPilot"/>
              <property name="jmsTemplateOrdersPilot" ref="jmsTemplateOrdersPilot"/>
              <property name="deadLetterQ" ref="deadLetterQ"/>
              <property name="utility" ref="messageParser"/>
              <property name="errorNotifier" ref="errorNotifier"/>
          </bean>
      
      
      
          <bean id="messageParser" class="com.my.company.util.AgentMessageUtility" init-method="loadPilotStores">
            <property name="pathToPilotStores" value="${path.to.pilot.stores}"/>
          </bean>
      
      
         <bean id="connectionFactory" class="com.ibm.mq.jms.MQConnectionFactory">
              <property name="channel" value="${mq.channel}"/>
              <property name="hostName" value="${mq.hostName}"/>
              <property name="port" value="${mq.port}"/>
              <property name="queueManager" value="${mq.queueManager}"/>
             <!-- JMSC.MQJMS_TP_CLIENT_MQ_TCPIP = 1-->
              <property name="transportType" value="${mq.transportType}"/>
             <property name="useConnectionPooling" value="true"/>
      
          </bean>
      
          <bean id="cachedConnectionFactory" class="org.springframework.jms.connection.CachingConnectionFactory">
              <property name="targetConnectionFactory" ref="connectionFactory"/>
              <property name="sessionCacheSize" value="10"/>
      
          </bean>
      
          <!-- Listeners -->
      	<bean id="mmsOrderMessageListener" class="com.my.company.OrderMessageListener">
               <property name="routingService" ref="messageRoutingService"/>
              <property name="errorNotifier" ref="errorNotifier"/>
          </bean>
      
      
          <bean id="messageRoutingService" class="com.my.company.service.MessageRoutingService">
              <property name="messageUtility" ref="messageParser"/>
              <property name="errorNotifier" ref="errorNotifier"/>
              <property name="templateBuilder" ref="jmsTemplateFactory"/>
          </bean>
      
      
           <!-- Orders Container -->
           <bean class="org.springframework.jms.listener.DefaultMessageListenerContainer">
              <property name="connectionFactory" ref="cachedConnectionFactory"/>
              <property name="destinationName" value="${mq.inboundOrdersQueue}"/>
              <property name="messageListener" ref="mmsOrderMessageListener"/>
              <property name="transactionManager" ref="txManager" />
           </bean>
      
      
      
          <bean id="mailSender" class="org.springframework.mail.javamail.JavaMailSenderImpl">
              <property name="host" value="${mail.host}"/>
          </bean>
      
          <bean id="errorNotifier" class="com.my.company.notification.ErrorNotifierImpl">
              <property name="mailSender" ref="mailSender"/>
              <property name="from" value="${mail.from}"/>
              <property name="to" value="${mail.to}"/>
          </bean>
      
      
          <bean id="txManager" class="org.springframework.jms.connection.JmsTransactionManager">
             <property name="connectionFactory" ref="connectionFactory" />
          </bean>
      
      
          <bean id="log4jInitialization"
      	class="org.springframework.beans.factory.config.MethodInvokingFactoryBean">
      		<property name="targetClass"
      			value="org.springframework.util.Log4jConfigurer" />
      		<property name="targetMethod" value="initLogging" />
      		<property name="arguments">
      			<list>
      				<value>resources/log4j.xml</value>
      			</list>
      		</property>
      	</bean>
      
      </beans>

      Comment


      • #4
        Maybe the problem was a firewall or some network applicance blocking those 10000 sockets connection attemps ?

        Adding a CachingConnectionFactory would solve that problem because the same connection is reused for all message sent.

        Do you still experiment this problem when sending 10000 or more messages ?

        Comment


        • #5
          Originally posted by beaupral View Post
          Maybe the problem was a firewall or some network applicance blocking those 10000 sockets connection attemps ?

          Adding a CachingConnectionFactory would solve that problem because the same connection is reused for all message sent.

          Do you still experiment this problem when sending 10000 or more messages ?
          I've tested this a couple of times and seems to be working.

          I have tested with varying message loads. For example, sending 5000, 8000, and 10000 messages to the inbound queue.
          Then fired up my application which was able to pull the messages and route them to the appropriate destination queue.

          It seems the changes I made fixed my initial issue of my application not being able to handle all 10,000 messages.

          Comment

          Working...
          X