Announcement Announcement Module
Collapse
No announcement yet.
Error while closing item reader Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • Error while closing item reader

    Hi All,

    I have an batch processing that registers the job and start the registered job in different threads(Each job with different parameters).
    All the threads processing successfully, jobs are completed but i am getting following exception in the logs.

    Why this error occurs?
    How come jobs are completed successfully?

    Please find the Already closed text file contain error logs

    I am using Spring batch 2.1
    Hibernate3
    sql server 2005.

    Thanks for your reply in advance.

  • #2
    You are running into synchronization issues. Use the JdbbcCursorReader as a delegate to a synchronized read in your Reader with saveState =false .
    May be the jobs are completing but I am sure your will not see all the rows...did you verify that ?
    By default mostly the reads are not thread safe.

    Comment


    • #3
      Thanks for your reply Sambaran.
      We are using default HibernateCursorItemReader. I have verified the data. There is to scenario's
      1. Some time Job is completed successfully and data also proper but exception we are getting in logs.
      2. Some time job is failed and data is improper.

      According your suggestion I will add the delegate and synchronized the read.

      One more doubts.
      We are getting Connection Already close and Connection Reset exception. What this exception occurs.

      We are getting this issues while running our batch in Production environment(AIX). Database size is around 750GB.
      Any other information required?
      Last edited by arun4; Mar 16th, 2012, 06:04 AM.

      Comment


      • #4
        I think what is happening is that One thread is already closing the connection without notifying others and the second thread comes in and tries to close the connection again. As the connection is already closed it bombs out.

        As I said before I am pretty sure a synchronized delegate with saveState =false is goign to solve the problem.
        saveState =false will help since then the reader is not going to try to save the read counts which is also not synchronized

        Comment


        • #5
          Thanks for your reply sambaran.
          We have started testing with synchronized delegate. Once done, we will update you.

          Comment


          • #6
            Thanks for your reply Sambaran.
            We are using default HibernateCursorItemReader. I have verified the data. There is to scenario's
            1. Some time Job is completed successfully and data also proper but exception we are getting in logs.
            nice information

            Comment


            • #7
              Hi Sambaran,
              We have tested with the synchronized delegate. we are getting same exception in the
              my.application.batch.core.reader.SynchronizedItemR eader.close(SynchronizedItemReader.java:54).

              Please find the SynchronizedItemReader for your reference.
              Code:
              ublic class SynchronizedItemReader <T> implements ItemReader <T>, ItemStream {
                 
                  /** The delegate. */
                  private ItemReader <T> delegate;
                 
                  /**
                   * Read.
                   *
                   * @throws Exception
                   *             the Exception
                   * @return T domain object
                   */
                  public synchronized T read() throws Exception {
              
                      return delegate.read();
                  }
                 
                  /**
                   * setDelegate.
                   *
                   * @param delegate
                   *            the item reader
                   */
                 
                  public void setDelegate(ItemReader <T> delegate) {
              
                      this.delegate = delegate;
                  }
                 
                  /**
                   * close.
                   *
                   */
                  public void close() {
              
                      if (this.delegate instanceof ItemStream) {
                          ((ItemStream) this.delegate).close();
                      }
                  }
                 
                  /**
                   * open.
                   *
                   * @param context
                   *            the context
                   */
                  public void open(ExecutionContext context) {
              
                      if (this.delegate instanceof ItemStream) {
                          ((ItemStream) this.delegate).open(context);
                      }
                  }
                 
                  /**
                   * update.
                   *
                   * @param context
                   *            the context
                   */
                  public void update(ExecutionContext context) {
              
                      if (this.delegate instanceof ItemStream) {
                          ((ItemStream) this.delegate).update(context);
                      }
                  }
              }

              Comment


              • #8
                Did you make sure that you are disabling the savestate (savestate=false) in the reader

                Comment


                • #9
                  Please find the SynchronizedItemReader bean code.

                  Code:
                  	<bean id="commonStepListItemReader"
                  		class="my.application.batch.core.reader.SynchronizedItemReader">
                  		<property name="delegate" ref="listItemReader" />
                  	</bean>
                  
                  	<bean id="listItemReader"
                  		class="org.springframework.batch.item.database.HibernateCursorItemReader"
                  		scope="step">
                  		<property name="sessionFactory">
                  			<ref bean="batchSessionFactory" />
                  		</property>
                  		<property name="queryString"
                  			value="from ListItemDo where listNumber=#{jobParameters['listNumber']} AND statusCode = 0">
                  		</property>
                  		<property name="saveState" value="false" />
                  	</bean>
                  Whenever we are getting already closed in the spring batch, we are also getting the JobInterruptedException. Please find the JobInterruptedException.txt attached.
                  Already closed and JobInterruptedException occured but batch job is completed. How it is possible? batch job should fail right? Please clarify the same.

                  thanks for your reply in advance.
                  Last edited by arun4; Mar 27th, 2012, 01:13 AM.

                  Comment


                  • #10
                    Are you trying to kill the job execution ..when it is running ?
                    When you job execution ran fine...does it mean all data in the databases are proper.did you verify ?
                    Also you are adding any debug statement append the current thread name as well...will be a little bit easier to debug.

                    Comment


                    • #11
                      We are not trying to kill the job execution. Data in the database is not proper. No all the item processed.
                      We had some test in local environment:
                      Test1: Run the same code (Same Database backups) in windows. No connection reset occurs.
                      Test2: Run The Same code (Same Database backups) in Linux. We are getting connection reset.

                      NOte: Production Environment is AIX.

                      Jre version is same for both windows and linux.
                      What could be the reasons?

                      Comment


                      • #12
                        Are you are still getting the reader closed error ?

                        What is the pooling mechanism you are using ?
                        What about transaction boundaries...can you make sure that you are sure you are using the same transaction across the whole step?

                        Comment


                        • #13
                          Yes we are getting the reader closed error.

                          This is my job configuration. In the processor we are using opensession and close session to processing the business.

                          Code:
                          <job id="Process_New_Plan_InstructionJob" restartable="true"
                          		parent="common_Abstract_ParentJob" xmlns="http://www.springframework.org/schema/batch">
                          		<!-- Process all pending instructions of a single plan. -->
                          		<step id="Process_New_Plan_InstructionStep">
                          			<tasklet allow-start-if-complete="true">
                          				<chunk reader="commonStepListItemReader" processor="batchRunTaskListProcessorWrapper"
                          					writer="commonStepListWriter" commit-interval="${commitInterval}"
                          					skip-limit="${skipLimit}" retry-limit="${retryLimit}">
                          					<skippable-exception-classes>
                          						<include
                          							class="my.application.batch.core.exception.EpsilonBatchBusinessException" />
                          						<include class="org.hibernate.exception.LockAcquisitionException" />
                          					</skippable-exception-classes>
                          					<retryable-exception-classes>
                          						<include class="org.hibernate.exception.LockAcquisitionException" />
                          					</retryable-exception-classes>
                          				</chunk>
                          				<listeners>
                          					<listener ref="stepListItemJobListener" />
                          					<listener ref="promotionListener" />
                          				</listeners>
                          			</tasklet>
                          		</step>

                          Comment


                          • #14
                            Arun I do not see any taskexecutor in your config..how are you implementing the mutithreading then ? how are you using your transaction manager...can you keep the transaction manager in the step level and try running the job.

                            Comment


                            • #15
                              Transaction manager is there in the step level. (Sorry copy paste mistake).

                              MultiThreading logic:
                              Jvm first load the job and as per business, we will split the records in to different list(Called BatchList). Each batchList contain number of records to process(Called BatchListItems). Note: We will not get duplicate records in the Items.

                              As per number of lists we have start 5 or 10 threads. Each thread will pick a list and process the items.

                              Comment

                              Working...
                              X