Announcement Announcement Module
Collapse
No announcement yet.
Use parameters in job.xml using %% Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • Use parameters in job.xml using %%

    Hi,
    I use m5 version of spring batch .
    I run my batch jobs with the parameters : /monjob.xml monjob batch.scheduleDate=200803111000.

    In my job.xml file, I would like to use the batch.scheduleDate value in the name of an output file.
    For that, I have replaced ${batch.scheduleDate} in the name of the ouputFile (in m3 version) by %batch.scheduleDate% but it doesn't work.
    Here is my job.xml file.
    <?xml version="1.0" encoding="UTF-8"?>
    <beans xmlns="http://www.springframework.org/schema/beans"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xmlns:aop="http://www.springframework.org/schema/aop"
    xsi:schemaLocation="http://www.springframework.org/schema/beans
    http://www.springframework.org/schem...-beans-2.0.xsd
    http://www.springframework.org/schema/aop
    http://www.springframework.org/schema/aop/spring-aop-2.0.xsd">

    <import resource="classpath:simple-container-definition.xml" />

    <bean id="ExtractionClientsJob" parent="simpleJob">
    <property name="steps">
    <list>
    <bean id="Extraction" parent="simpleStep">
    <property name="streams" ref="clientFileWriter"/>
    <property name="commitInterval" value="2" />
    <property name="allowStartIfComplete" value="false" />
    <property name="itemReader" ref="DbItemReader" />
    <property name="itemWriter" ref="clientFileWriter"/>
    </bean>
    </list>
    </property>
    </bean>

    <!-- INFRASTRUCTURE SETUP -->
    <bean id="DbItemReader" class="com.natixis.sphinx.batch.reader.DbItemReade r">
    <property name="dataSource" ref="dataSource" />
    <property name="sql">
    <value> select p.ceseqo, p.cocivi,p.nompat,p.nommar,p.prenom,p.preno2,p.dah eme,p.cosecu,p.cositf,a.librue,a.copost,a.libvil,a .copays,a.coposn,a.copayn,a.libvin,c.cochar,c.dahe m1,c.numrib from sphmclient c, sphmadresse a, sphmpersonne p where c.cochar=7 and c.copers=p.ceseqo and p.ceseqo=a.copers order by p.ceseqo</value>
    </property>
    <property name="mapper">
    <bean class="com.natixis.sphinx.batch.modele.mapping.Cli entRowMapper" />
    </property>
    </bean>

    <bean id="clientFileWriter" class="org.springframework.batch.io.file.FlatFileI temWriter">
    <property name="resource" ref="resourceSortie1" />
    <property name="lineAggregator" >
    <bean class="org.springframework.batch.io.file.transform .DelimitedLineAggregator">
    <property name="delimiter" value=";"/>
    </bean>
    </property>
    <property name="fieldSetUnmapper" >
    <bean class="com.natixis.sphinx.batch.modele.mapping.Cli entFieldSetCreator" />
    </property>
    <property name="shouldDeleteIfExists" value="true"/>
    </bean>

    <bean id="mapContext" class="com.natixis.sphinx.batch.domain.Context">
    <property name="map">
    <map>
    <entry key="resourceSortie1" value-ref="resourceSortie1"/>
    <entry key="dataSource" value-ref="dataSource"/>
    </map>
    </property>
    </bean>


    <bean id="resourceSortie1" class="java.lang.String">
    <constructor-arg value="file:data/output/%batch.scheduleDate%.clients.txt"/>
    </bean>

    </beans>
    I have read that the class org.springframework.batch.execution.resource.StepE xecutionProxyResource replace %% by the value of parameters but I don't know what I have to do to be able to use it.

    Could you give me a sample which shows how to use parameters of batch in job.xml?

    Thanks in advance (sorry for my bad english).

  • #2
    I think you mean
    Code:
    %{batch.scheduleDate}

    Comment


    • #3
      I have tried:
      %{batch.scheduleDate}
      and
      %{batch.scheduleDate}%

      but it isn't replaced by the value.

      Comment


      • #4
        You need to use the actual StepExecutionResourceProxy, you're just creating a String. It should look something like:

        Code:
        <bean id="resourceSortie1" class="org.springframework.batch.core.support.StepExecutionResourceProxy">
          <property name="filePattern" value="data/output/%batch.scheduleDate%.clients.txt" />
        </bean>
        I haven't checked that to ensure it's valid, but I think you get the idea. The spring container won't automatically convert your string using the proxy, you have to declare it directly.

        Comment


        • #5
          I almost forget, you need to register the proxy with the step as well, so that it can get access to the job parameters.

          Comment


          • #6
            Thanks for your answers.

            I tried the following job.xml:
            <bean id="ExtractionClientsJob" parent="simpleJob">
            <property name="steps">
            <list>
            <bean id="Extraction" parent="simpleStep">
            <property name="listeners" ref="resourceSortie1"/> <property name="commitInterval" value="2" />
            <property name="allowStartIfComplete" value="false" />
            <property name="itemReader" ref="DBInputTemplate" />
            <property name="itemWriter" ref="clientFileWriter"/>

            </bean>
            </list>
            </property>
            </bean>
            ...
            <bean id="resourceSortie1" class="org.springframework.batch.execution.resource.StepE xecutionProxyResource">
            <property name="filePattern" value="data/output/%batch.scheduleDate%.clients.txt"/> </bean>
            But the error is always:
            The delegate resource has not been initialised. Remember to register this object as a StepListener.
            And I don't understand how to register this object as a StepListener.
            Could you clarify this point please?

            Comment


            • #7
              You have registered the listener in the fragement you showed. Maybe we are missing something else? From the stack trace you can probably see where the resource is being used - my guess it is being used outside the step somewhere?

              Comment


              • #8
                Yes it's used by clientFileWriter which is a FlatFileItemWriter:
                <bean id="clientFileWriter" class="org.springframework.batch.io.file.FlatFileI temWriter">
                <property name="resource" ref="resourceSortie1"/>
                <property name="lineAggregator" >
                <bean class="org.springframework.batch.io.file.transform .DelimitedLineAggregator">
                <property name="delimiter" value=";"/>
                </bean>
                </property>
                <property name="fieldSetUnmapper" >
                <bean class="com.natixis.sphinx.batch.modele.mapping.Cli entFieldSetCreator" />
                </property>
                <property name="shouldDeleteIfExists" value="true"/>
                </bean>
                But a flatFileItemWriter isn't a stepListener and can't register stepListener. Then I don't know what I must do.

                Comment


                • #9
                  You need to register the proxy as a StepListener, separate from the writer.

                  Comment


                  • #10
                    Sorry but I don't understand the code I must write to register the proxy.

                    Could you give me the code I must write in my job.xml or explain what I must do exactly (perhaps code in job.xml is good and I must create a itemReader class which implements ItemReadListener and then use my itemReader class instead of FlatFileItemReader)?

                    Comment


                    • #11
                      I don't have time to look back at M5, but the latest snapshot has a simple setListeners method. Something like:

                      Code:
                      defaultStepFactoryBean.setListeners(StepExecutionResourceProxy);
                      Of course this would be in xml and not code, but I think you get the idea. I believe in M5 you would need to set that directly on the ItemOrientedStep.
                      Last edited by lucasward; Mar 12th, 2008, 12:23 PM. Reason: fixed code tags

                      Comment


                      • #12
                        Originally posted by sandrine View Post
                        Yes it's used by clientFileWriter which is a FlatFileItemWriter
                        And is that file writer used somewhere outside a step? The stack trace might help.

                        Comment


                        • #13
                          here is the log:
                          ERROR - BatchMain.start(170) | Job Terminated in error:
                          org.springframework.beans.factory.BeanCreationExce ption: Error creating bean with name 'ExtractionClientsJob' defined in class path resource [jobs/ExtractionClientsJob.xml]: Cannot create inner bean 'Extraction' of type [org.springframework.batch.execution.step.support.D efaultStepFactoryBean] while setting bean property 'steps' with key [0]; nested exception is org.springframework.beans.factory.BeanCreationExce ption: Error creating bean with name 'Extraction' defined in class path resource [jobs/ExtractionClientsJob.xml]: Cannot resolve reference to bean 'clientFileWriter' while setting bean property 'itemWriter'; nested exception is org.springframework.beans.factory.BeanCreationExce ption: Error creating bean with name 'clientFileWriter' defined in class path resource [jobs/ExtractionClientsJob.xml]: Invocation of init method failed; nested exception is java.lang.IllegalStateException: The delegate resource has not been initialised. Remember to register this object as a StepListener.
                          at org.springframework.beans.factory.support.BeanDefi nitionValueResolver.resolveInnerBean(BeanDefinitio nValueResolver.java:229)
                          at org.springframework.beans.factory.support.BeanDefi nitionValueResolver.resolveValueIfNecessary(BeanDe finitionValueResolver.java:117)
                          at org.springframework.beans.factory.support.BeanDefi nitionValueResolver.resolveManagedList(BeanDefinit ionValueResolver.java:286)
                          at org.springframework.beans.factory.support.BeanDefi nitionValueResolver.resolveValueIfNecessary(BeanDe finitionValueResolver.java:126)
                          at org.springframework.beans.factory.support.Abstract AutowireCapableBeanFactory.applyPropertyValues(Abs tractAutowireCapableBeanFactory.java:1274)
                          at org.springframework.beans.factory.support.Abstract AutowireCapableBeanFactory.populateBean(AbstractAu towireCapableBeanFactory.java:1042)
                          at org.springframework.beans.factory.support.Abstract AutowireCapableBeanFactory.doCreateBean(AbstractAu towireCapableBeanFactory.java:539)
                          at org.springframework.beans.factory.support.Abstract AutowireCapableBeanFactory$1.run(AbstractAutowireC apableBeanFactory.java:485)
                          at java.security.AccessController.doPrivileged(Access Controller.java:197)
                          at org.springframework.beans.factory.support.Abstract AutowireCapableBeanFactory.createBean(AbstractAuto wireCapableBeanFactory.java:455)
                          at org.springframework.beans.factory.support.Abstract BeanFactory$1.getObject(AbstractBeanFactory.java:2 51)
                          at org.springframework.beans.factory.support.DefaultS ingletonBeanRegistry.getSingleton(DefaultSingleton BeanRegistry.java:169)
                          at org.springframework.beans.factory.support.Abstract BeanFactory.getBean(AbstractBeanFactory.java:248)
                          at org.springframework.beans.factory.support.Abstract BeanFactory.getBean(AbstractBeanFactory.java:170)
                          at org.springframework.beans.factory.support.DefaultL istableBeanFactory.preInstantiateSingletons(Defaul tListableBeanFactory.java:413)
                          at org.springframework.context.support.AbstractApplic ationContext.finishBeanFactoryInitialization(Abstr actApplicationContext.java:735)
                          at org.springframework.context.support.AbstractApplic ationContext.refresh(AbstractApplicationContext.ja va:369)
                          at org.springframework.context.support.ClassPathXmlAp plicationContext.<init>(ClassPathXmlApplicationCon text.java:122)
                          at org.springframework.context.support.ClassPathXmlAp plicationContext.<init>(ClassPathXmlApplicationCon text.java:66)
                          at com.natixis.sphinx.batch.BatchMain.start(BatchMain .java:146)
                          at com.natixis.sphinx.batch.BatchMain.main(BatchMain. java:212)
                          Caused by:
                          org.springframework.beans.factory.BeanCreationExce ption: Error creating bean with name 'Extraction' defined in class path resource [jobs/ExtractionClientsJob.xml]: Cannot resolve reference to bean 'clientFileWriter' while setting bean property 'itemWriter'; nested exception is org.springframework.beans.factory.BeanCreationExce ption: Error creating bean with name 'clientFileWriter' defined in class path resource [jobs/ExtractionClientsJob.xml]: Invocation of init method failed; nested exception is java.lang.IllegalStateException: The delegate resource has not been initialised. Remember to register this object as a StepListener.
                          at org.springframework.beans.factory.support.BeanDefi nitionValueResolver.resolveReference(BeanDefinitio nValueResolver.java:274)
                          at org.springframework.beans.factory.support.BeanDefi nitionValueResolver.resolveValueIfNecessary(BeanDe finitionValueResolver.java:104)
                          at org.springframework.beans.factory.support.Abstract AutowireCapableBeanFactory.applyPropertyValues(Abs tractAutowireCapableBeanFactory.java:1274)
                          at org.springframework.beans.factory.support.Abstract AutowireCapableBeanFactory.populateBean(AbstractAu towireCapableBeanFactory.java:1042)
                          at org.springframework.beans.factory.support.Abstract AutowireCapableBeanFactory.doCreateBean(AbstractAu towireCapableBeanFactory.java:539)
                          at org.springframework.beans.factory.support.Abstract AutowireCapableBeanFactory$1.run(AbstractAutowireC apableBeanFactory.java:485)
                          at java.security.AccessController.doPrivileged(Access Controller.java:197)
                          at org.springframework.beans.factory.support.Abstract AutowireCapableBeanFactory.createBean(AbstractAuto wireCapableBeanFactory.java:455)
                          at org.springframework.beans.factory.support.BeanDefi nitionValueResolver.resolveInnerBean(BeanDefinitio nValueResolver.java:219)
                          ... 20 more
                          Caused by:
                          org.springframework.beans.factory.BeanCreationExce ption: Error creating bean with name 'clientFileWriter' defined in class path resource [jobs/ExtractionClientsJob.xml]: Invocation of init method failed; nested exception is java.lang.IllegalStateException: The delegate resource has not been initialised. Remember to register this object as a StepListener.
                          at org.springframework.beans.factory.support.Abstract AutowireCapableBeanFactory.initializeBean(Abstract AutowireCapableBeanFactory.java:1362)
                          at org.springframework.beans.factory.support.Abstract AutowireCapableBeanFactory.doCreateBean(AbstractAu towireCapableBeanFactory.java:540)
                          at org.springframework.beans.factory.support.Abstract AutowireCapableBeanFactory$1.run(AbstractAutowireC apableBeanFactory.java:485)
                          at java.security.AccessController.doPrivileged(Access Controller.java:197)
                          at org.springframework.beans.factory.support.Abstract AutowireCapableBeanFactory.createBean(AbstractAuto wireCapableBeanFactory.java:455)
                          at org.springframework.beans.factory.support.Abstract BeanFactory$1.getObject(AbstractBeanFactory.java:2 51)
                          at org.springframework.beans.factory.support.DefaultS ingletonBeanRegistry.getSingleton(DefaultSingleton BeanRegistry.java:169)
                          at org.springframework.beans.factory.support.Abstract BeanFactory.getBean(AbstractBeanFactory.java:248)
                          at org.springframework.beans.factory.support.Abstract BeanFactory.getBean(AbstractBeanFactory.java:170)
                          at org.springframework.beans.factory.support.BeanDefi nitionValueResolver.resolveReference(BeanDefinitio nValueResolver.java:268)
                          ... 28 more
                          Caused by:
                          java.lang.IllegalStateException: The delegate resource has not been initialised. Remember to register this object as a StepListener.
                          at org.springframework.util.Assert.state(Assert.java: 355)
                          at org.springframework.batch.execution.resource.StepE xecutionProxyResource.getFile(StepExecutionProxyRe source.java:132)
                          at org.springframework.batch.io.file.FlatFileItemWrit er.afterPropertiesSet(FlatFileItemWriter.java:96)
                          at org.springframework.beans.factory.support.Abstract AutowireCapableBeanFactory.invokeInitMethods(Abstr actAutowireCapableBeanFactory.java:1390)
                          at org.springframework.beans.factory.support.Abstract AutowireCapableBeanFactory.initializeBean(Abstract AutowireCapableBeanFactory.java:1359)
                          ... 37 more
                          I use a DefaultStepFactoryBean which has a listeners property like ItemOrientedStep has stepListeners property.

                          I have seen that in StepExecutionProxyResource there is a field named "delegate" which must not be null but there isn't setDelegate method in this class and so it is always null!!!

                          If I am wrong, could you say how to set the delegate?
                          Else could you add the setDelegate method in the next version?

                          I think I'm going to wait for the next version of spring Batch to use parameter of batch instead of system's parameter.

                          Sorry I should have had to see this problem before.

                          Comment


                          • #14
                            Delegate is set when the step starts and calls the beforeStep(..) callback, it's ok there is no setter for it.

                            Comment


                            • #15
                              This works just fine.

                              Here is the job:

                              <bean id="fixedLengthFaresAssessorToBlsJob" parent="simpleJob">

                              <property name="steps">

                              <bean id="stepReadFaresAssessorAndWriteToBls" parent="simpleStep">
                              <property name="listeners">
                              <list>
                              <ref bean="fileResource"/>
                              </list>
                              </property>
                              <!-- need a stream to open record before we read it in -->

                              <property name="streams" ref="faresAssessorFlatFileItemReader"/>
                              <property name="itemReader" ref="faresAssessorToBlsItemReader"/>
                              <property name="itemWriter" ref="blsWriter"/>
                              </bean>
                              </property>
                              </bean>


                              and resource:

                              <bean id="fileResource" class="org.springframework.batch.execution.resourc e.StepExecutionProxyResource">
                              <property name="filePattern" value="data/%file.name%"/>
                              </bean>

                              <bean id="faresAssessorFlatFileItemReader" class="org.springframework.batch.io.file.FlatFileI temReader">
                              <property name="resource" ref="fileResource" />
                              <property name="lineTokenizer" ref="faresAssessorFixedFileDescriptor" />
                              <property name="fieldSetMapper">
                              <bean class="org.springframework.batch.io.file.mapping.P assThroughFieldSetMapper" />
                              </property>
                              </bean>


                              Then just add job parameters as such:

                              jpb.addString("file.name", "fixedLengthFaresAssessorToBlsSmallData.txt");
                              launcher.run(job,jpb.toJobParameters());


                              Thanks.

                              Franz Garsombke

                              Comment

                              Working...
                              X