Announcement Announcement Module
Collapse
No announcement yet.
Trying to configure Spring Batch Job Repository Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • Trying to configure Spring Batch Job Repository

    I'm trying to setup a Spring Batch project from scratch to understand how the plumbing fits together (as opposed to using an eclipse / Spring Batch template project).

    I've followed this series of tutorials upto this point http://numberformat.wordpress.com/20...ob-repository/ .

    Where I'm getting stuck is trying to configure the internal job repository to work with my database (Postgres in my case). I've done this in the past by tweaking working spring batch template projects and I feel the approach presented in the tutorial is somewhat cumbersome (specifying DDLs , initializer classes etc ..., as opposed to using inbuilt Spring functionality).

    However, when I attempt to execute a simple batch program, I get the following error. Looking at the DB, the tables and sequences are created (verified by wiping them and re-executing the program) but it appears that there is a duplicate attempt to create them (that fails).

    Thanks in advance for any help. The XML config should be visible when the moderator approves their posting
    Last edited by vybe3142; Nov 9th, 2011, 09:19 AM.

  • #2
    Error Message
    Code:
    vinay@proton ~/workspaces/defaultworkspace/springBatchHelloWorld $ mvn clean compile exec:java -Dexec.mainClass=org.springframework.batch.core.launch.support.CommandLineJobRunner -Dexec.args="simpleJob.xml helloWorldJob"
    [INFO] Scanning for projects...
    [INFO] Searching repository for plugin with prefix: 'exec'.
    [INFO] ------------------------------------------------------------------------
    [INFO] Building springBatchHelloWorld
    [INFO]    task-segment: [clean, compile, exec:java]
    [INFO] ------------------------------------------------------------------------
    [INFO] [clean:clean {execution: default-clean}]
    [INFO] Deleting file set: /home/vinay/workspaces/defaultworkspace/springBatchHelloWorld/target (included: [**], excluded: [])
    [INFO] [resources:resources {execution: default-resources}]
    [INFO] Using 'UTF-8' encoding to copy filtered resources.
    [INFO] Copying 4 resources
    [INFO] [compiler:compile {execution: default-compile}]
    [INFO] Compiling 5 source files to /home/vinay/workspaces/defaultworkspace/springBatchHelloWorld/target/classes
    [INFO] Preparing exec:java
    [INFO] No goals needed for project - skipping
    [INFO] [exec:java {execution: default-cli}]
    Nov 9, 2011 9:07:31 AM org.springframework.context.support.AbstractApplicationContext prepareRefresh
    INFO: Refreshing org.springframework.context.support.ClassPathXmlApplicationContext@acaf083: display name [org.springframework.context.support.ClassPathXmlApplicationContext@acaf083]; startup date [Wed Nov 09 09:07:31 CST 2011]; root of context hierarchy
    Nov 9, 2011 9:07:31 AM org.springframework.beans.factory.xml.XmlBeanDefinitionReader loadBeanDefinitions
    INFO: Loading XML bean definitions from class path resource [simpleJob.xml]
    Nov 9, 2011 9:07:31 AM org.springframework.beans.factory.xml.XmlBeanDefinitionReader loadBeanDefinitions
    INFO: Loading XML bean definitions from class path resource [applicationContext.xml]
    Nov 9, 2011 9:07:31 AM org.springframework.beans.factory.support.DefaultListableBeanFactory registerBeanDefinition
    INFO: Overriding bean definition for bean 'helloWorldJob': replacing [Generic bean: class [org.springframework.batch.core.configuration.xml.SimpleFlowFactoryBean]; scope=; abstract=false; lazyInit=false; autowireMode=0; dependencyCheck=0; autowireCandidate=true; primary=false; factoryBeanName=null; factoryMethodName=null; initMethodName=null; destroyMethodName=null] with [Generic bean: class [org.springframework.batch.core.configuration.xml.JobParserJobFactoryBean]; scope=; abstract=false; lazyInit=false; autowireMode=0; dependencyCheck=0; autowireCandidate=true; primary=false; factoryBeanName=null; factoryMethodName=null; initMethodName=null; destroyMethodName=null]
    Nov 9, 2011 9:07:31 AM org.springframework.context.support.AbstractApplicationContext obtainFreshBeanFactory
    INFO: Bean factory for application context [org.springframework.context.support.ClassPathXmlApplicationContext@acaf083]: org.springframework.beans.factory.support.DefaultListableBeanFactory@520b1684
    Nov 9, 2011 9:07:31 AM org.springframework.core.io.support.PropertiesLoaderSupport loadProperties
    INFO: Loading properties file from class path resource [batch.properties]
    Nov 9, 2011 9:07:31 AM org.springframework.beans.factory.support.DefaultListableBeanFactory preInstantiateSingletons
    INFO: Pre-instantiating singletons in org.springframework.beans.factory.support.DefaultListableBeanFactory@520b1684: defining beans [jobRepository,jobLauncher,transactionManager,dataSource,org.springframework.beans.factory.config.PropertyPlaceholderConfigurer#0,org.springframework.jdbc.datasource.init.DataSourceInitializer#0,defaultTokenizer,employeeFieldSetMapper,employeeLineMapper,empReader,empProcessor,empWriter,empHeaderFooterWriter,org.springframework.batch.core.scope.internalStepScope,org.springframework.beans.factory.config.CustomEditorConfigurer,org.springframework.batch.core.configuration.xml.CoreNamespacePostProcessor,step1,step2,helloWorldJob,helloWorldTasklet]; root of factory hierarchy
    Nov 9, 2011 9:07:32 AM org.springframework.batch.core.launch.support.SimpleJobLauncher afterPropertiesSet
    INFO: No TaskExecutor has been set, defaulting to synchronous executor.
    Nov 9, 2011 9:07:32 AM org.springframework.jdbc.datasource.init.ResourceDatabasePopulator executeSqlScript
    INFO: Executing SQL script from class path resource [org/springframework/batch/core/schema-postgresql.sql]
    Nov 9, 2011 9:07:32 AM org.springframework.beans.factory.support.DefaultSingletonBeanRegistry destroySingletons
    INFO: Destroying singletons in org.springframework.beans.factory.support.DefaultListableBeanFactory@520b1684: defining beans [jobRepository,jobLauncher,transactionManager,dataSource,org.springframework.beans.factory.config.PropertyPlaceholderConfigurer#0,org.springframework.jdbc.datasource.init.DataSourceInitializer#0,defaultTokenizer,employeeFieldSetMapper,employeeLineMapper,empReader,empProcessor,empWriter,empHeaderFooterWriter,org.springframework.batch.core.scope.internalStepScope,org.springframework.beans.factory.config.CustomEditorConfigurer,org.springframework.batch.core.configuration.xml.CoreNamespacePostProcessor,step1,step2,helloWorldJob,helloWorldTasklet]; root of factory hierarchy
    Nov 9, 2011 9:07:32 AM org.springframework.batch.core.launch.support.CommandLineJobRunner start
    SEVERE: Job Terminated in error: Error creating bean with name 'org.springframework.jdbc.datasource.init.DataSourceInitializer#0': Invocation of init method failed; nested exception is org.springframework.dao.DataAccessResourceFailureException: Failed to populate database; nested exception is org.springframework.jdbc.datasource.init.ScriptStatementFailedException: Failed to execute SQL script statement at line 1 of resource class path resource [org/springframework/batch/core/schema-postgresql.sql]: CREATE TABLE BATCH_JOB_INSTANCE  (  JOB_INSTANCE_ID BIGINT  NOT NULL PRIMARY KEY ,    VERSION BIGINT ,    JOB_NAME VARCHAR(100) NOT NULL,   JOB_KEY VARCHAR(32) NOT NULL,  constraint JOB_INST_UN unique (JOB_NAME, JOB_KEY) ) 
    org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'org.springframework.jdbc.datasource.init.DataSourceInitializer#0': Invocation of init method failed; nested exception is org.springframework.dao.DataAccessResourceFailureException: Failed to populate database; nested exception is org.springframework.jdbc.datasource.init.ScriptStatementFailedException: Failed to execute SQL script statement at line 1 of resource class path resource [org/springframework/batch/core/schema-postgresql.sql]: CREATE TABLE BATCH_JOB_INSTANCE  (  JOB_INSTANCE_ID BIGINT  NOT NULL PRIMARY KEY ,    VERSION BIGINT ,    JOB_NAME VARCHAR(100) NOT NULL,   JOB_KEY VARCHAR(32) NOT NULL,  constraint JOB_INST_UN unique (JOB_NAME, JOB_KEY) ) 
    	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1420)
    	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:519)
    	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:456)
    	at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:293)
    	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222)
    	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:290)
    	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:192)
    	at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:585)
    	at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:728)
    	at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:380)
    	at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:139)
    	at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:83)
    	at org.springframework.batch.core.launch.support.CommandLineJobRunner.start(CommandLineJobRunner.java:282)
    	at org.springframework.batch.core.launch.support.CommandLineJobRunner.main(CommandLineJobRunner.java:574)
    	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    	at java.lang.reflect.Method.invoke(Method.java:597)
    	at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:297)
    	at java.lang.Thread.run(Thread.java:662)
    Caused by: org.springframework.dao.DataAccessResourceFailureException: Failed to populate database; nested exception is org.springframework.jdbc.datasource.init.ScriptStatementFailedException: Failed to execute SQL script statement at line 1 of resource class path resource [org/springframework/batch/core/schema-postgresql.sql]: CREATE TABLE BATCH_JOB_INSTANCE  (  JOB_INSTANCE_ID BIGINT  NOT NULL PRIMARY KEY ,    VERSION BIGINT ,    JOB_NAME VARCHAR(100) NOT NULL,   JOB_KEY VARCHAR(32) NOT NULL,  constraint JOB_INST_UN unique (JOB_NAME, JOB_KEY) ) 
    	at org.springframework.jdbc.datasource.init.DataSourceInitializer.afterPropertiesSet(DataSourceInitializer.java:92)
    	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1477)
    	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1417)
    	... 19 more
    .......

    Comment


    • #3
      My relevant pom.xml and spring config files follow

      Code:
      <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
      	xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
      	<modelVersion>4.0.0</modelVersion>
      
      	<groupId>com.test</groupId>
      	<artifactId>springBatchHelloWorld</artifactId>
      	<version>1.0-SNAPSHOT</version>
      	<packaging>jar</packaging>
      
      	<name>springBatchHelloWorld</name>
      	<url>http://maven.apache.org</url>
      
      	<properties>
      		<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
      		<spring.framework.jdbc.version>3.0.6.RELEASE</spring.framework.jdbc.version>
      		<spring.batch.version>2.1.7.RELEASE</spring.batch.version>
      	</properties>
      
      	<dependencies>
      		<dependency>
      			<groupId>junit</groupId>
      			<artifactId>junit</artifactId>
      			<version>3.8.1</version>
      			<scope>test</scope>
      		</dependency>
      
      		<dependency>
      			<groupId>org.springframework</groupId>
      			<artifactId>spring-jdbc</artifactId>
      			<version>${spring.framework.jdbc.version}</version>
      		</dependency>
      
      		<dependency>
      			<groupId>org.springframework</groupId>
      			<artifactId>spring-aop</artifactId>
      			<version>${spring.framework.jdbc.version}</version>
      		</dependency>
      
      		<dependency>
      			<groupId>commons-dbcp</groupId>
      			<artifactId>commons-dbcp</artifactId>
      			<version>1.2.2</version>
      		</dependency>
      
      		<dependency>
      			<groupId>postgresql</groupId>
      			<artifactId>postgresql</artifactId>
      			<version>9.0-801.jdbc4</version>
      		</dependency>
      
      		<dependency>
      			<groupId>org.springframework.batch</groupId>
      			<artifactId>spring-batch-core</artifactId>
      			<version>${spring.batch.version}</version>
      		</dependency>
      		<dependency>
      			<groupId>commons-lang</groupId>
      			<artifactId>commons-lang</artifactId>
      			<version>2.1</version>
      		</dependency>
      		<dependency>
      			<groupId>org.springframework</groupId>
      			<artifactId>spring</artifactId>
      			<version>2.5.6</version>
      		</dependency>
      
      
      	</dependencies>
      
      	<build>
      		<plugins>
      			<plugin>
      				<groupId>org.apache.maven.plugins</groupId>
      				<artifactId>maven-compiler-plugin</artifactId>
      				<version>2.0.2</version>
      				<configuration>
      					<source>1.5</source>
      					<target>1.5</target>
      				</configuration>
      			</plugin>
      		</plugins>
      	</build>
      
      </project>

      Code:
      <?xml version="1.0" encoding="UTF-8"?>
      <beans:beans xmlns="http://www.springframework.org/schema/batch"
      	xmlns:beans="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
      	xmlns:context="http://www.springframework.org/schema/context"
      	xmlns:jdbc="http://www.springframework.org/schema/jdbc"
      	xsi:schemaLocation="
      http://www.springframework.org/schema/beans
      http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
      http://www.springframework.org/schema/batch
      http://www.springframework.org/schema/batch/spring-batch-2.1.xsd
      http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-3.0.xsd
      http://www.springframework.org/schema/jdbc http://www.springframework.org/schema/jdbc/spring-jdbc-3.0.xsd
      ">
      
      <!-- 	<beans:bean id="jobRepository"
      		class="org.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean">
      		<beans:property name="transactionManager" ref="transactionManager" />
      	</beans:bean> -->
      
      <beans:bean id="jobRepository"
          class="org.springframework.batch.core.repository.support.JobRepositoryFactoryBean">
          <beans:property name="databaseType" value="POSTGRES"/>
          <beans:property name="dataSource" ref="dataSource"/>
          <beans:property name="transactionManager" ref="transactionManager" />
      </beans:bean>
      
      
      
      
      
      	<beans:bean id="jobLauncher"
      		class="org.springframework.batch.core.launch.support.SimpleJobLauncher">
      		<beans:property name="jobRepository" ref="jobRepository" />
      	</beans:bean>
      
      	<beans:bean id="transactionManager"
      		class="org.springframework.batch.support.transaction.ResourcelessTransactionManager" />
      
      
      	<!-- -->
      <!-- 	<beans:bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource"
      		destroy-method="close">
      		<beans:property name="driverClassName" value="org.postgresql.Driver" />
      		<beans:property name="url" value="jdbc:postgresql://localhost/test" />
      		<beans:property name="username" value="postgres" />
      		<beans:property name="password" value="postgressux" />
      	</beans:bean>
       -->
      	<beans:bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource"
      		destroy-method="close">
      		<beans:property name="driverClassName" value="${batch.jdbc.driver}" />
      		<beans:property name="url" value="${batch.jdbc.url}" />
      		<beans:property name="username" value="${batch.jdbc.user}" />
      		<beans:property name="password" value="${batch.jdbc.password}" />
      	</beans:bean>
      	
      	<context:property-placeholder location="classpath:batch.properties" />
      
      	<jdbc:initialize-database data-source="dataSource">
      		<jdbc:script location="${batch.schema.script}" />
      	</jdbc:initialize-database>
      
      
      	<!-- -->
      </beans:beans>
      Code:
      <?xml version="1.0" encoding="UTF-8"?>
      <beans:beans xmlns="http://www.springframework.org/schema/batch"
      	xmlns:beans="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
      	xsi:schemaLocation="
      
      http://www.springframework.org/schema/batch
      
      http://www.springframework.org/schema/batch/spring-batch-2.1.xsd
      
      http://www.springframework.org/schema/beans
      
      http://www.springframework.org/schema/beans/spring-beans-2.0.xsd">
      
      	<beans:import resource="applicationContext.xml" />
      
      	<!-- Tokenizer - Converts a delimited string into a Set of Fields -->
      	<beans:bean name="defaultTokenizer"
      		class="org.springframework.batch.item.file.transform.DelimitedLineTokenizer" />
      
      	<!-- FieldSetMapper - Populates a bean's attributes with using the FieldSet -->
      	<beans:bean name="employeeFieldSetMapper" class="com.test.EmployeeFieldSetMapper" />
      
      	<!-- LineMapper - Uses the tokenizer and Mapper to create instances of a 
      		Bean. -->
      	<beans:bean name="employeeLineMapper"
      		class="org.springframework.batch.item.file.mapping.DefaultLineMapper">
      		<beans:property name="lineTokenizer" ref="defaultTokenizer" />
      		<beans:property name="fieldSetMapper" ref="employeeFieldSetMapper" />
      	</beans:bean>
      
      	<!-- Reader - used by the tasklet to process one Item from the input. -->
      	<beans:bean name="empReader"
      		class="org.springframework.batch.item.file.FlatFileItemReader">
      		<beans:property name="lineMapper" ref="employeeLineMapper" />
      
      		<!-- use spring integrations for the following, but for now filename is 
      			hard coded -->
      		<beans:property name="resource" value="input_data.txt" />
      	</beans:bean>
      
      	<!-- Processor -->
      
      	<beans:bean name="empProcessor" class="com.test.EmployeeProcessor">
      	</beans:bean>
      
      	<!-- Writer -->
      	<beans:bean id="empWriter"
      		class="org.springframework.batch.item.file.FlatFileItemWriter">
      		<beans:property name="resource" value="file:target/output_data.txt" />
      		<beans:property name="lineAggregator">
      			<beans:bean
      				class="org.springframework.batch.item.file.transform.DelimitedLineAggregator">
      				<beans:property name="delimiter" value="|" />
      				<beans:property name="fieldExtractor">
      					<beans:bean
      						class="org.springframework.batch.item.file.transform.BeanWrapperFieldExtractor">
      						<beans:property name="names"
      							value="empId,lastName,title,salary,rank" />
      					</beans:bean>
      				</beans:property>
      			</beans:bean>
      		</beans:property>
      		<beans:property name="footerCallback" ref="empHeaderFooterWriter" />
      		<beans:property name="headerCallback" ref="empHeaderFooterWriter" />
      	</beans:bean>
      
      	<beans:bean id="empHeaderFooterWriter" class="com.test.EmployeeItemWriter">
      		<beans:property name="delegate" ref="empWriter" />
      	</beans:bean>
      
      
      	<job id="helloWorldJob">
      		<step id="step1" next="step2">
      			<tasklet ref="helloWorldTasklet" />
      		</step>
      		<step id="step2">
      			<tasklet>
      <!-- 				<chunk reader="empReader" processor="empProcessor" writer="empWriter"
      					commit-interval="1" /> -->
      					
      					<!-- Use the modified writer -->
                  <chunk reader="empReader" processor="empProcessor" writer="empHeaderFooterWriter" commit-interval="1"/>					
      			</tasklet>
      		</step>
      	</job>
      
      	<beans:bean name="helloWorldTasklet" class="com.test.HelloWorldTasklet" />
      
      	<!-- To run the job from the command line type the following: mvn clean 
      		compile exec:java -Dexec.mainClass=org.springframework.batch.core.launch.support.CommandLineJobRunner 
      		-Dexec.args="simpleJob.xml helloWorldJob" -->
      </beans:beans>

      Code:
      batch.jdbc.url=jdbc:postgresql://localhost/my_springbatch_db
      batch.jdbc.driver=org.postgresql.Driver
      batch.jdbc.user=postgres
      batch.jdbc.password=****************
      batch.jdbc.testWhileIdle=false
      batch.jdbc.validationQuery=
      batch.drop.script=/org/springframework/batch/core/schema-drop-postgresql.sql
      batch.schema.script=/org/springframework/batch/core/schema-postgresql.sql
      batch.business.schema.script=business-schema-postgresql.sql
      batch.data.source.init=true
      batch.database.incrementer.class=org.springframework.jdbc.support.incrementer.PostgreSQLSequenceMaxValueIncrementer
      batch.lob.handler.class=org.springframework.jdbc.support.lob.DefaultLobHandler
      batch.database.incrementer.parent=sequenceIncrementerParent
      batch.grid.size=2
      batch.verify.cursor.position=true

      Comment


      • #4
        Is the SQL script that Spring is trying to execute working fine ? Have you verified by executing script from SQL client ?

        Comment

        Working...
        X