Announcement Announcement Module
Collapse
No announcement yet.
Execution Context issue Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • Execution Context issue

    I'm having trouble executing a sample spring batch job. Looks like the problem is that the step execution context is not being saved to the batch framework table (step_execution_context). The short context and serialized context columns are always null. The code is throwing an exception when its trying to getLastStepExecution() in SimpleJobRepository object

    DEBUG main org.springframework.jdbc.core.JdbcTemplate - Executing prepared SQL statement [SELECT SHORT_CONTEXT, SERIALIZED_CONTEXT FROM BATCH_STEP_EXECUTION_CONTEXT WHERE STEP_EXECUTION_ID = ?]
    org.json.JSONException: A JSONObject text must begin with '{' at character 1 of of
    at org.json.JSONTokener.syntaxError(JSONTokener.java: 448)
    at org.json.JSONObject.<init>(JSONObject.java:178)
    at org.codehaus.jettison.mapped.MappedXMLInputFactory .createXMLStreamReader(MappedXMLInputFactory.java: 39)
    at org.codehaus.jettison.AbstractXMLInputFactory.crea teXMLStreamReader(AbstractXMLInputFactory.java:132 )
    at com.thoughtworks.xstream.io.json.JettisonMappedXml Driver.createReader(JettisonMappedXmlDriver.java:5 2)
    at com.thoughtworks.xstream.XStream.fromXML(XStream.j ava:861)
    at com.thoughtworks.xstream.XStream.fromXML(XStream.j ava:853)
    at org.springframework.batch.core.repository.dao.XStr eamExecutionContextStringSerializer.deserialize(XS treamExecutionContextStringSerializer.java:48)
    at org.springframework.batch.core.repository.dao.Jdbc ExecutionContextDao$ExecutionContextRowMapper.mapR ow(JdbcExecutionContextDao.java:204)
    at org.springframework.batch.core.repository.dao.Jdbc ExecutionContextDao$ExecutionContextRowMapper.mapR ow(JdbcExecutionContextDao.java:1)
    at org.springframework.jdbc.core.RowMapperResultSetEx tractor.extractData(RowMapperResultSetExtractor.ja va:92)
    at org.springframework.jdbc.core.JdbcTemplate$1.doInP reparedStatement(JdbcTemplate.java:653)
    at org.springframework.jdbc.core.JdbcTemplate.execute (JdbcTemplate.java:591)
    at org.springframework.jdbc.core.JdbcTemplate.query(J dbcTemplate.java:641)
    at org.springframework.jdbc.core.JdbcTemplate.query(J dbcTemplate.java:670)
    at org.springframework.jdbc.core.JdbcTemplate.query(J dbcTemplate.java:678)
    at org.springframework.jdbc.core.JdbcTemplate.query(J dbcTemplate.java:710)
    at org.springframework.jdbc.core.simple.SimpleJdbcTem plate.query(SimpleJdbcTemplate.java:187)
    at org.springframework.batch.core.repository.dao.Jdbc ExecutionContextDao.getExecutionContext(JdbcExecut ionContextDao.java:90)
    at org.springframework.batch.core.repository.support. SimpleJobRepository.getLastStepExecution(SimpleJob Repository.java:206)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Nativ e Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(Native MethodAccessorImpl.java:79)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(De legatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:618)
    at org.springframework.aop.support.AopUtils.invokeJoi npointUsingReflection(AopUtils.java:307)
    at org.springframework.aop.framework.ReflectiveMethod Invocation.invokeJoinpoint(ReflectiveMethodInvocat ion.java:182)
    at org.springframework.aop.framework.ReflectiveMethod Invocation.proceed(ReflectiveMethodInvocation.java :149)
    at org.springframework.transaction.interceptor.Transa ctionInterceptor.invoke(TransactionInterceptor.jav a:106)
    at org.springframework.aop.framework.ReflectiveMethod Invocation.proceed(ReflectiveMethodInvocation.java :171)
    at org.springframework.aop.framework.JdkDynamicAopPro xy.invoke(JdkDynamicAopProxy.java:204)
    at $Proxy4.getLastStepExecution(Unknown Source)
    at com.jpmorgan.rps.poc.item.PortfolioExecutionSplitt er.getStartable(PortfolioExecutionSplitter.java:16 0)
    at com.jpmorgan.rps.poc.item.PortfolioExecutionSplitt er.split(PortfolioExecutionSplitter.java:131)
    at com.jpmorgan.rps.poc.multisponsor.SampleTaskExecut orPartitionHandler.handle(SampleTaskExecutorPartit ionHandler.java:82)
    at org.springframework.batch.core.partition.support.P artitionStep.doExecute(PartitionStep.java:91)
    at org.springframework.batch.core.step.AbstractStep.e xecute(AbstractStep.java:198)
    at org.springframework.batch.core.step.item.SimpleChu nkProcessor.writeItems(SimpleChunkProcessor.java:1 56)
    at org.springframework.batch.core.step.item.SimpleChu nkProcessor.doWrite(SimpleChunkProcessor.java:137)
    at org.springframework.batch.core.step.item.SimpleChu nkProcessor.write(SimpleChunkProcessor.java:252)
    at org.springframework.batch.core.step.item.SimpleChu nkProcessor.process(SimpleChunkProcessor.java:178)
    at org.springframework.batch.core.step.item.ChunkOrie ntedTasklet.execute(ChunkOrientedTasklet.java:74)
    at org.springframework.batch.core.step.tasklet.Taskle tStep$2.doInChunkContext(TaskletStep.java:268)
    at org.springframework.batch.core.scope.context.StepC ontextRepeatCallback.doInIteration(StepContextRepe atCallback.java:76)
    at org.springframework.batch.repeat.support.RepeatTem plate.getNextResult(RepeatTemplate.java:367)
    at org.springframework.batch.repeat.support.RepeatTem plate.executeInternal(RepeatTemplate.java:215)
    at org.springframework.batch.repeat.support.RepeatTem plate.iterate(RepeatTemplate.java:143)
    at org.springframework.batch.core.step.tasklet.Taskle tStep.doExecute(TaskletStep.java:242)
    at org.springframework.batch.core.step.AbstractStep.e xecute(AbstractStep.java:198)
    at org.springframework.batch.core.partition.support.T askExecutorPartitionHandler$1.call(TaskExecutorPar titionHandler.java:108)
    at org.springframework.batch.core.partition.support.T askExecutorPartitionHandler$1.call(TaskExecutorPar titionHandler.java:1)
    at java.util.concurrent.FutureTask$Sync.innerRun(Futu reTask.java:284)
    at java.util.concurrent.FutureTask.run(FutureTask.jav a:138)
    at org.springframework.core.task.SyncTaskExecutor.exe cute(SyncTaskExecutor.java:49)
    at org.springframework.batch.core.partition.support.T askExecutorPartitionHandler.handle(TaskExecutorPar titionHandler.java:114)
    at org.springframework.batch.core.partition.support.P artitionStep.doExecute(PartitionStep.java:91)
    at org.springframework.batch.core.step.AbstractStep.e xecute(AbstractStep.java:198)
    at org.springframework.batch.core.job.AbstractJob.han dleStep(AbstractJob.java:348)
    at org.springframework.batch.core.job.flow.FlowJob.ac cess$0(FlowJob.java:1)
    at org.springframework.batch.core.job.flow.FlowJob$Jo bFlowExecutor.executeStep(FlowJob.java:135)
    at org.springframework.batch.core.job.flow.support.st ate.StepState.handle(StepState.java:60)
    at org.springframework.batch.core.job.flow.support.Si mpleFlow.resume(SimpleFlow.java:144)
    at org.springframework.batch.core.job.flow.support.Si mpleFlow.start(SimpleFlow.java:124)
    at org.springframework.batch.core.job.flow.FlowJob.do Execute(FlowJob.java:103)
    at org.springframework.batch.core.job.AbstractJob.exe cute(AbstractJob.java:250)
    at org.springframework.batch.core.launch.support.Simp leJobLauncher$1.run(SimpleJobLauncher.java:110)
    at org.springframework.core.task.SyncTaskExecutor.exe cute(SyncTaskExecutor.java:49)
    at org.springframework.batch.core.launch.support.Simp leJobLauncher.run(SimpleJobLauncher.java:105)
    at org.springframework.batch.test.AbstractJobTests.la unchJob(AbstractJobTests.java:139)
    at com.jpmorgan.rps.xxx.MultiReaderJobTest.testJob(Mu ltiReaderJobTest.java:59)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Nativ e Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(Native MethodAccessorImpl.java:79)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(De legatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:618)
    at org.springframework.test.context.junit4.SpringTest Method.invoke(SpringTestMethod.java:160)
    at org.springframework.test.context.junit4
    (ClassRoadie.java:27)
    at org.junit.internal.runners.ClassRoadie.runProtecte d(ClassRoadie.java:37)
    at org.junit.internal.runners.JUnit4ClassRunner.run(J Unit4ClassRunner.java:42)
    at org.springframework.test.context.junit4.SpringJUni t4ClassRunner.run(SpringJUnit4ClassRunner.java:97)
    at org.eclipse.jdt.internal.junit4.runner.JUnit4TestR eference.run(JUnit4TestReference.java:38)
    at org.eclipse.jdt.internal.junit.runner.TestExecutio n.run(TestExecution.java:38)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRu nner.runTests(RemoteTestRunner.java:460)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRu nner.runTests(RemoteTestRunner.java:673)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRu nner.run(RemoteTestRunner.java:386)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRu nner.main(RemoteTestRunner.java:196)

    I did some debug through the spring batch framework code and looks like the contents of the step execution context object are not being saved to the BATCH_STEP_EXECUTION_CONTEXT table.
    I'm putting String name value pairs into the step context object, but the
    serializer.serialize(m) always returns empty String object and a null value is being inserted to the db table.
    The serializer being used is XStreamExecutionContextStringSerializer and the batch framework tables are created in DB2 UDB.
    Could you provide any pointers to why the execution context is not being saved to the batch framework tables?

  • #2
    Not much detail here really. Maybe you could post the code you are using to create the execution context entries and some of your configuration for the job and/or step?

    Comment


    • #3
      code

      Config files and code for the patition splitter has been attached. Please let me know if you need any other files

      Comment


      • #4
        Is this happening on a restart? Or a fresh launch of a new job execution?

        Aside: from what I can see, creating a StepExecutionSplitter is not the way I would have implemented this use case. You should be able to use a Partitioner with some listener and late binding tricks to get the information you want at runtime in the step executions.

        Comment


        • #5
          Dave,
          Thanks for the reply. This happens on a fresh start of the job execution.

          Also, could you elaborate on using partitioner and listeners with late binding.

          The approach i have used is the SimplePartitioner uses the splitter to create separate execution contexts and run each step in separate thread using the ThreadExecutor. The approach is similar to Partition usecase. I'm new to this framework so may be i'm wrong. But could you please provide more details on the approach you would follow to design the following use case
          1. Get the list of records from database (ItemReader)
          2. Iterate through the record set
          3. Read 5 records (from ItemReader) at a time and run each record in a separate thread.
          4. Each record updates the results to database
          5. Read next set of 5 records from ItemReader, continue processing

          Comment


          • #6
            Your use case as described sounds pretty vanilla. Why do you need each item in the group of 5 to be processed in a separate thread? Would it be enough to process chunks of 5 in parallel (that's what a standard <tasklet/> with task-executor set in the XML would do)? If you need to write to a file or other single-threaded stateful resource, then this might not be restartable, but it should at least work, and if it does then at least we have something less obscure to analyse.

            If you need to use a partitioning approach then I would try to use the SimpleStepExecutionSplitter in a regular PartitionStep and TaskExecutorPartitionHandler. Late binding in step scope is a general concept and is described in the user guide.

            Comment


            • #7
              Regarding the execution context not being saved to database tables, i created a sample custom serializer class and looks like it started saving the context to the table. But it doesn't work with the xml serializer used in the spring framework.

              To ur question regarding multiple threads, each item needs further processing to generate a String which is in turn written to a file. So to be more specific,
              We have a high level partition say has 100 items and for each of these 100 items another query retrieves 10000 sub items. So my thought was to use multithreading to parallel process these sub items. And as per the documentation looks like it can be achieved with the PatitionStep.

              Comment


              • #8
                Originally posted by vinodnarayanan View Post
                Regarding the execution context not being saved to database tables, i created a sample custom serializer class and looks like it started saving the context to the table. But it doesn't work with the xml serializer used in the spring framework.
                That's odd. Can you give some examples of tthe data that were being stored? I thought you said it was just Strings. Can you make a test case that fails with a simple usage of JdbcExecutionContextDao?

                To ur question regarding multiple threads, each item needs further processing to generate a String which is in turn written to a file. So to be more specific,
                We have a high level partition say has 100 items and for each of these 100 items another query retrieves 10000 sub items. So my thought was to use multithreading to parallel process these sub items. And as per the documentation looks like it can be achieved with the PatitionStep.
                PartitionStep sounds perfect. But you shouldn't need to implement StepExecutionSplitter unless I'm missing something. A Partitioner should suffice - and as a bonus you get a more natural configuration API in the XML (with 2.1).

                Comment


                • #9
                  I tried running the XStreamExecutionContextStringSerializerTests but junit test cases fails because serialize method always returns a null String.

                  I'm using xstream-1.3.jar

                  Attached is the junit test class.

                  The exception is
                  org.json.JSONException: A JSONObject text must begin with '{' at character 1 of
                  at org.json.JSONTokener.syntaxError(JSONTokener.java: 448)
                  at org.json.JSONObject.<init>(JSONObject.java:178)
                  at org.codehaus.jettison.mapped.MappedXMLInputFactory .createXMLStreamReader(MappedXMLInputFactory.java: 39)
                  at org.codehaus.jettison.AbstractXMLInputFactory.crea teXMLStreamReader(AbstractXMLInputFactory.java:132 )
                  at com.thoughtworks.xstream.io.json.JettisonMappedXml Driver.createReader(JettisonMappedXmlDriver.java:5 2)
                  at com.thoughtworks.xstream.XStream.fromXML(XStream.j ava:861)
                  at com.thoughtworks.xstream.XStream.fromXML(XStream.j ava:853)
                  at org.springframework.batch.core.repository.dao.XStr eamExecutionContextStringSerializer.deserialize(XS treamExecutionContextStringSerializer.java:48)
                  at com.jpmorgan.rps.xxx.JdbcExecutionContextDao.testS erializeAMap(JdbcExecutionContextDao.java:43)
                  at sun.reflect.NativeMethodAccessorImpl.invoke0(Nativ e Method)
                  at sun.reflect.NativeMethodAccessorImpl.invoke(Native MethodAccessorImpl.java:79)
                  at sun.reflect.DelegatingMethodAccessorImpl.invoke(De legatingMethodAccessorImpl.java:43)
                  at java.lang.reflect.Method.invoke(Method.java:618)
                  at junit.framework.TestCase.runTest(TestCase.java:168 )
                  at junit.framework.TestCase.runBare(TestCase.java:134 )
                  at junit.framework.TestResult$1.protect(TestResult.ja va:110)
                  at junit.framework.TestResult.runProtected(TestResult .java:128)
                  at junit.framework.TestResult.run(TestResult.java:113 )
                  at junit.framework.TestCase.run(TestCase.java:124)
                  at junit.framework.TestSuite.runTest(TestSuite.java:2 32)
                  at junit.framework.TestSuite.run(TestSuite.java:227)
                  at org.junit.internal.runners.JUnit38ClassRunner.run( JUnit38ClassRunner.java:81)
                  at org.eclipse.jdt.internal.junit4.runner.JUnit4TestR eference.run(JUnit4TestReference.java:38)
                  at org.eclipse.jdt.internal.junit.runner.TestExecutio n.run(TestExecution.java:38)
                  at org.eclipse.jdt.internal.junit.runner.RemoteTestRu nner.runTests(RemoteTestRunner.java:460)
                  at org.eclipse.jdt.internal.junit.runner.RemoteTestRu nner.runTests(RemoteTestRunner.java:673)
                  at org.eclipse.jdt.internal.junit.runner.RemoteTestRu nner.run(RemoteTestRunner.java:386)
                  at org.eclipse.jdt.internal.junit.runner.RemoteTestRu nner.main(RemoteTestRunner.java:196)

                  Comment


                  • #10
                    Hello,
                    Any thoughts on this?

                    Comment


                    • #11
                      What version of Jettison are you using?

                      Comment


                      • #12
                        jettison-1.0-alpha-1.jar

                        Comment


                        • #13
                          Hi. I ran the test and got the same issue using 2.0.3.RELEASE code.

                          When is this expected to be fixed? Do we need to open a bug report?

                          Comment


                          • #14
                            I don't think it's a bug yet since everything works for everyone else. Open a JIRA if you think it is, but we will still need to get to the bottom of all this. Can you try upgrading to Jettison 1.1?

                            Comment


                            • #15
                              Thanks Dave, the issue got resolved after i upgraded to jettison 1.1

                              Also, is there any place where it lists all the recommended version of dependent jar files to be used for a release.

                              Comment

                              Working...
                              X