Announcement Announcement Module
Collapse
No announcement yet.
Transaction isolation Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • Transaction isolation

    Hi all,

    My question is related to transaction isolation for batch updates in spring-batch.
    I would like to set custom transaction isolation on a transaction supplied by spring-batch. How can it be done? I use a StatefulRetryStepFactoryBean and a SimpleJob classes for my job definition.

    Thanks in advance.

    Alex

  • #2
    Hi,
    Please refer retry-transaction-test.xml which comes with the downloads, for changing the transaction isolation.

    Comment


    • #3
      org.springframework.batch.core.step.item.ItemOrien tedStep.execute opens transaction programaticaly. My questoion was - how can I change its isolation?

      Thanks.

      Comment


      • #4
        You're right, there isn't a way to provide the isolation to use for the transaction manager that's wired into the ItemOrientedStep. I suppose we could add that as a configuration option, but I'm curious what your use case is?

        Comment


        • #5
          About my Use case

          Lucas, thank you for your answer.
          I'll try to describe my use case as far as I can.
          We ported our code to 1.0.0.FINAL.
          We have a process that listens to a socket and receives data from a sending process. The sending process (it is in a different JVM) accumulates data from several thousand sensors.
          Our listener parses the received data and writes it to 14 different files (as per sensor type). Then 14 jobs read these files, every job knows what files to read (according to file name pattern). For every sensor type there is a corresponding table in the database, plus one table that represents sensors hierarchy.
          All 14 tables have a foreign key constraint on the hierarchy table.
          And now the main part.
          A job reads a file and tries to insert the data to the corresponding table.
          When the job inserts data from a sensor that is not present in the hierarchy table, and fails on the foreign key constraint. In the RetryListener.onError() there is enough information for it to add the new sensor into the hierarchy and then retry the insert. The sensor addition to the hierarchy table is performed via a Hibernate Dao that is wrapped in a service, so that the service is defined with @Transactional(propagation=Propagation.REQUIRES_NE W). All the data insertion to the child table is performed via iBatis.

          My problem is in the Hibernate call to the service. It is stuck as the surrounding iBatis transaction locks the table (we use MySQL InnoDb for all the tables).
          We thought that by changing transaction isolation level on the spring-batch transactions (that run iBatis Daos) we will be able to eliminate the lock.
          Also we use JpaTransactionManager.

          As a workaround I used a different thread for the hierarchy table update, and that worked, but it's a partial and ugly solution, that may fail, I know.
          Here is my RetryListener code
          Code:
          public class StepRetryListener extends RetryListenerSupport {
          
          	private ReporterService reporterService;
          
          	private ReporterType reporterType;
          
          	@Autowired
          	public void setReporterService(ReporterService reporterService) {
          		this.reporterService = reporterService;
          	}
          
          	public void setReporterType(ReporterType reporterType) {
          		this.reporterType = reporterType;
          	}
          
          	@Override
          	public void onError(RetryContext context, RetryCallback callback, Throwable throwable) {
          
          		Object data = ((ItemWriterRetryCallback)callback).getItem();
          
          		String errorMessage = throwable.getMessage();
          
          		if (StringUtils.containsIgnoreCase(errorMessage, Constants.FOREIGN_KEY_CONSTRAINT)) {
          			// A reference constaint error (unknown reporter sent data)
          			// 1) insert a new record into hierarchy table
          			HashMap map = (HashMap)data;
          
          			Integer senderId = (Integer)map.get(BaseFieldSetMapper.SENDER_ID);
          			if (senderId != null) {
          
          				Long reporterId = (Long)map.get(BaseFieldSetMapper.REPORTER_ID);
          				// via Hibernate 
          				Field field = reporterService.getFieldBySenderId(senderId);
          
          				final Reporter reporter = new Reporter();
          				reporter.setField(field);
          				reporter.setSenderId(senderId);
          				reporter.setReporterId(reporterId);
          				reporter.setReporterType(reporterType);
          // must be here
          //reporterService.saveReporter(reporter);
          
          				// this is done in a new transaction
          				Executors.newSingleThreadExecutor().submit(new Callable<Object>() {
          					public Object call() throws Exception {
          						//via Hibernate
          						reporterService.saveReporter(reporter);
          						return null;
          					}
          				});
          
          			}
          		}
          
          	}
          
          }
          Hope, that there is a more elegant solution for the problem.
          About the same problem I wrote in the different thread http://forum.springframework.org/sho...843#post173843 and that's why I asked if it is possible to run retry after recover, as updating the hierarchy table in a recoverer doesn't impose the table lock, the transactions a re totally isolated, and not one inside the other.

          I'm looking forward to seeing your answer or tip. As the previous time with 1.0.0.m3 your answer helped me.
          Thank you in advance.

          Alex.
          Last edited by alexbt; Apr 10th, 2008, 02:01 AM. Reason: explanation addition

          Comment


          • #6
            You could change the isolation level at the DataSource level. That works well enough in a lot of cases like this. At least you will be able to test if it helps to change the isolation level, until we expose that option in the step configuration. Hint: use IsolationLevelDataSourceAdapter.

            Comment


            • #7
              About your hint

              Thank you Dave for the answer.
              Unfortunatelly, I don't see how I can use this adapter in our configuration. We use JpaTransactionManager, and our server runs on Jetty. As far as I understand the usage of IsolationLevelDataSourceAdapter demands utilization of JtaTransactionManager, that is not our case.
              Do you have any other suggestion? Or may be I'm wrong.

              Thank you in advance.
              Alex

              Comment


              • #8
                I don't think there's any restriction on the transaction manager. You just inject a DataSource.

                Comment


                • #9
                  The connection has a method setTransactionIsolation, which is what the adaptor is working on. Regardless of where the connection comes from you should be able to set this. I don't think the transaction manager overrides it either unless you specify it in the TransactionDefinition (which I don't believe we do)

                  Comment


                  • #10
                    Thanks

                    Lucas and Dave,
                    Thank you very much for your help. You did direct me to the right place with your tips. We found a workaround for our issue. But for the future, wouldn't it be good to have a transaction isolation level configurable?!
                    May be you should judge it as a feature request for version 1.1.0.

                    If it is present in the version, it will solve our problem and we won't need the workaround.

                    Regards,
                    Alex

                    Comment


                    • #11
                      It sounds like an excellent idea for 1.1. If you open a JIRA, you will be able to track when the feature is available.

                      Comment

                      Working...
                      X