Announcement Announcement Module
Collapse
No announcement yet.
Problem using SimpleAsyncTaskExecutor Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • Problem using SimpleAsyncTaskExecutor

    Hi,

    I was using spring batch 1.1.4 and i was using SimpleAsyncTaskExecutor to configure the taskExecutor for a SimpleStepFactoryBean.

    The throttleLimit was configured for 10 threads.

    My definition was bellow:

    <bean id="processItemInvoiceStages" parent="simpleStep">
    <property name="taskExecutor">
    <bean
    class="org.springframework.core.task.SimpleAsyncTa skExecutor">
    </bean>
    </property>
    <property name="itemReader" ref="processItemStageReader" />
    <property name="itemWriter" ref="rulesWriter" />
    <property name="commitInterval" value="1" />
    </bean>

    I set the commitInterval for 1, but i guess that it must be 10 too.

    The problem is that when i execute the jstack -l <pid> for my java process,
    nine threads are WAITING and only one as RUNNABLE. My process was slow probably because that behavior.
    My item writer insert records on a oracle database table.

    Someone know what can be the problem ? Thank you

    The output from jstack is: (for the WAITING THREAD)

    "SimpleAsyncTaskExecutor-1819" prio=10 tid=0x5db7f000 nid=0x6e8e waiting on condition [0x5d5fe000..0x5d5ff030]
    java.lang.Thread.State: WAITING (parking)
    at sun.misc.Unsafe.park(Native Method)
    - parking to wait for <0x6463d148> (a java.util.concurrent.Semaphore$NonfairSync)
    at java.util.concurrent.locks.LockSupport.park(LockSu pport.java:158)
    at java.util.concurrent.locks.AbstractQueuedSynchroni zer.parkAndCheckInterrupt(AbstractQueuedSynchroniz er.java:747)
    at java.util.concurrent.locks.AbstractQueuedSynchroni zer.doAcquireSharedInterruptibly(AbstractQueuedSyn chronizer.java:905)
    at java.util.concurrent.locks.AbstractQueuedSynchroni zer.acquireSharedInterruptibly(AbstractQueuedSynch ronizer.java:1217)
    at java.util.concurrent.Semaphore.acquire(Semaphore.j ava:286)
    at org.springframework.batch.core.step.JdkConcurrentS tepExecutionSynchronizer.lock(JdkConcurrentStepExe cutionSynchronizer.java:33)
    at org.springframework.batch.core.step.item.ItemOrien tedStep$2.doInIteration(ItemOrientedStep.java:267)
    at org.springframework.batch.repeat.support.TaskExecu torRepeatTemplate$ExecutingRunnable.run(TaskExecut orRepeatTemplate.java:215)
    at java.lang.Thread.run(Thread.java:619)

    Locked ownable synchronizers:
    - None

    .
    .
    .


    AND IT for the RUNNABLE thread:

    "SimpleAsyncTaskExecutor-1810" prio=10 tid=0x5c40c400 nid=0x63bd runnable [0x5cfad000..0x5cfadfb0]
    java.lang.Thread.State: RUNNABLE
    at java.net.SocketInputStream.socketRead0(Native Method)
    at java.net.SocketInputStream.read(SocketInputStream. java:129)
    at oracle.net.ns.Packet.receive(Unknown Source)
    at oracle.net.ns.DataPacket.receive(Unknown Source)
    at oracle.net.ns.NetInputStream.getNextPacket(Unknown Source)
    at oracle.net.ns.NetInputStream.read(Unknown Source)
    at oracle.net.ns.NetInputStream.read(Unknown Source)
    at oracle.net.ns.NetInputStream.read(Unknown Source)
    at oracle.jdbc.driver.T4CMAREngine.unmarshalUB1(T4CMA REngine.java:1099)
    at oracle.jdbc.driver.T4CMAREngine.unmarshalSB1(T4CMA REngine.java:1070)
    at oracle.jdbc.driver.T4C8Oall.receive(T4C8Oall.java: 478)
    at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4 CPreparedStatement.java:216)
    at oracle.jdbc.driver.T4CPreparedStatement.executeFor Rows(T4CPreparedStatement.java:955)
    at oracle.jdbc.driver.OraclePreparedStatement.execute Batch(OraclePreparedStatement.java:10580)
    - locked <0x6ffd72b0> (a oracle.jdbc.driver.T4CPreparedStatement)
    - locked <0x6fa7a2d8> (a oracle.jdbc.driver.T4CConnection)
    at org.apache.commons.dbcp.DelegatingStatement.execut eBatch(DelegatingStatement.java:297)
    at org.apache.commons.dbcp.DelegatingStatement.execut eBatch(DelegatingStatement.java:297)
    at org.hibernate.jdbc.BatchingBatcher.doExecuteBatch( BatchingBatcher.java:48)
    at org.hibernate.jdbc.BatchingBatcher.addToBatch(Batc hingBatcher.java:34)
    at org.hibernate.persister.entity.AbstractEntityPersi ster.insert(AbstractEntityPersister.java:2247)
    at org.hibernate.persister.entity.AbstractEntityPersi ster.insert(AbstractEntityPersister.java:2660)
    at org.hibernate.action.EntityInsertAction.execute(En tityInsertAction.java:52)
    at org.hibernate.engine.ActionQueue.execute(ActionQue ue.java:248)
    at org.hibernate.engine.ActionQueue.executeActions(Ac tionQueue.java:232)
    at org.hibernate.engine.ActionQueue.executeActions(Ac tionQueue.java:139)
    at org.hibernate.event.def.AbstractFlushingEventListe ner.performExecutions(AbstractFlushingEventListene r.java:298)
    at org.hibernate.event.def.DefaultFlushEventListener. onFlush(DefaultFlushEventListener.java:27)
    at org.hibernate.impl.SessionImpl.flush(SessionImpl.j ava:1000)
    at org.hibernate.impl.SessionImpl.managedFlush(Sessio nImpl.java:338)
    at org.hibernate.transaction.JDBCTransaction.commit(J DBCTransaction.java:106)
    at org.springframework.orm.hibernate3.HibernateTransa ctionManager.doCommit(HibernateTransactionManager. java:655)
    at org.springframework.transaction.support.AbstractPl atformTransactionManager.processCommit(AbstractPla tformTransactionManager.java:709)
    at org.springframework.transaction.support.AbstractPl atformTransactionManager.commit(AbstractPlatformTr ansactionManager.java:678)
    at org.springframework.batch.core.step.item.ItemOrien tedStep$2.doInIteration(ItemOrientedStep.java:296)
    at org.springframework.batch.repeat.support.TaskExecu torRepeatTemplate$ExecutingRunnable.run(TaskExecut orRepeatTemplate.java:215)
    at java.lang.Thread.run(Thread.java:619)

  • #2
    With Batch 1.1 (up to 2.1) you have to be careful to make sure your DataSource connection pool has enough connections to service all the concurrent processing. If your Step is the only client of the DataSource at that time it would need 10 or more connections.

    Comment


    • #3
      Hi Dave,

      My datasource configuration is:


      <bean id="dataSource" class="com.cpqd.billing.ebppbatch.consolidation.da tasource.DataSourceWrapper">
      <property name="driverClassName" value="${batch.jdbc.driver}" />
      <property name="url" value="${batch.jdbc.url}" />
      <property name="username" value="${batch.jdbc.user}" />
      <property name="password" value="${batch.jdbc.password}" />
      <property name="initialSize" value="30" />
      <property name="poolPreparedStatements" value="true"/>
      <property name="maxActive" value="30" />
      <property name="clientName" value="${batch.client.name}" />
      <property name="databaseProperties" value="etc/batch-${environment}.properties" />
      </bean>



      I have 30 active connections configure.

      I guess that the problem can be the commitInterval, because it was configure with 1, but i guess that it must be 10 too.
      The other threads that are WAITING,
      are waiting the commit if the other thread.

      If someone have a sugestion for that problem i will stay happy.

      Comment


      • #4
        There is no connection between the commit interval and the concurrency settings (throttle limit, thread pool size, connection pool size).

        Your RUNNING thread is busy communicating with the database server. Do you think it is hung (nothing in the stack you posted suggests a problem). Maybe the server has a session limit or something?

        Comment


        • #5
          Hi Dave,

          I was using hibernate for insert many records on a database table. (about 18.000.000 records or more)
          I was using hibernate-3.2.3 and my hibernate.cfg.xml was configured with;

          <property name="hibernate.jdbc.batch_size">1000</property>

          Do you guess that it can be a performance hibernate problem on session flush (i was using no stateless session) ?
          Maybe must I use pure jdbc for insert records ?

          thank you.

          Comment


          • #6
            I have the same problems with you. I dont know why it appears. There are many unknown.....................
            -------------------------------------------------------------------------------------
            Web Development Company
            Last edited by ltm0807; Sep 14th, 2010, 10:48 PM.

            Comment


            • #7
              If your job works with the default task executor then there can be no issue with hibernate or batch sizes there. Was the process only slow?

              Just re-read the original post. If commitInterval=1 then you get no benefit from the async task executor because you spend all your time waiting for a synchronized update to the step execution. What about using a larger commit interval?

              Comment


              • #8
                Hi Dave,

                Really the threads that were WAITING, are waiting the commit of all the data inserted (i guess that the transaction was opened by spring batch).
                I will try open a transaction (REQUIRES_NEW) on my itemWriter (call it on write method), to force a commit for each thread.

                I post a new message if it will function.

                Thank you

                Comment


                • #9
                  What about using a larger commit interval?
                  That is the right solution if you want parallel processing to do anything useful.
                  Last edited by Dave Syer; Sep 17th, 2010, 03:05 AM. Reason: copy-paste error

                  Comment

                  Working...
                  X