Announcement Announcement Module
Collapse
No announcement yet.
Custom FlatFileItemReader Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • Custom FlatFileItemReader

    Hi,

    I'm extending spring frameworks FlatFileItemReader to assign a unique name to the flat file that will be written to the disk by appending a timestamp.

    However, the file generated does not have this unique name and new content gets appended into the existing file. What's wrong?

    Code:
    	@Override
    	public void setResource(Resource resource) {
    		String url = createURL(resource);
    		if(url!=null)
    			this.resource=new FileSystemResource(url);
    		else
    			this.resource = resource;
    		super.setResource(resource);
    	}
    
            //createURL method takes the resource argument and adds a timestamp to the filename.
    
            // Debug statements shows that the createURL() method is working and a unique filename is generated.

  • #2
    Has anyone implemented such a use case?

    Comment


    • #3
      FlatFileItemReader doesn't write or create any files, so I assume that's a typo?

      You can do simple things like this with SpEL, e.g. something like this (not tested so look up the SpEL stuff in the Spring user guide if it doesn't work).

      Code:
      <bean id="writer" class="...FlatFileItemWriter">
         <property name="resource" value="file://.../myfile-#{T (java.lang.System.currentTimeMillis())}.txt"/>
          ...
      </bean>

      Comment


      • #4
        You are right, it was a typo. In fact, I'm extending FlatFileItemWriter<T>.

        Comment


        • #5
          Thank you for your help.

          I guess extending FlatFileItemWriter is not a solution to dynamically generating a unique filename because the setter will be called only once when the bean is instantiated. So, I will have to look for some other solution.

          Even if, I try the SpEL expression that will also not give a unique name each time the job is executed?

          Still, looking for answer.....
          Last edited by tiger.spring; Apr 27th, 2011, 10:22 AM. Reason: premature posting...

          Comment


          • #6
            Now, I'm trying to use job parameters to specify a filename for the resource property as follows:

            Code:
            <bean class="org.springframework.batch.item.file.FlatFileItemWriter">
            	<property name="resource" value="#{jobParameters[input.filename]}"/>
            	<property name="lineAggregator">
            		<bean class="org.springframework.batch.item.file.transform.PassThroughLineAggregator" />
            	</property>
            </bean>
            However, I get following exception

            Code:
            Caused by: org.springframework.expression.spel.SpelEvaluationException: EL1008E:(pos 0): Field or property 'jobParameters' cannot be found on object of type 'org.springframework.beans.factory.config.BeanExpressionContext'
            	at org.springframework.expression.spel.ast.PropertyOrFieldReference.readProperty(PropertyOrFieldReference.java:207)
            	at org.springframework.expression.spel.ast.PropertyOrFieldReference.getValueInternal(PropertyOrFieldReference.java:71)
            	at org.springframework.expression.spel.ast.CompoundExpression.getValueInternal(CompoundExpression.java:52)
            	at org.springframework.expression.spel.ast.SpelNodeImpl.getValue(SpelNodeImpl.java:93)
            	at org.springframework.expression.spel.standard.SpelExpression.getValue(SpelExpression.java:88)
            	at org.springframework.context.expression.StandardBeanExpressionResolver.evaluate(StandardBeanExpressionResolver.java:138)
            	... 90 more

            Comment


            • #7
              I was missing scope="step" hence the above error. It works fine after that.

              However one strange issue, if a jobParameter is specified as "output.filename" it looks for getter/setter for property named output and if you remove the "." and make it "outputfilename" it works fine.

              Not sure if the property should have a period in it or not?

              Code:
              <property name="resource" value="#{jobParameters[output.filename]}" />
              
              Caused by: org.springframework.beans.NotReadablePropertyException: Invalid property 'output' of bean class [org.springframework.batch.core.scope.context.StepContext]: Bean property 'output' is not readable or has an invalid getter method: Does the return type of the getter match the parameter type of the setter?
              	at org.springframework.beans.BeanWrapperImpl.getPropertyValue(BeanWrapperImpl.java:707)
              	at org.springframework.beans.BeanWrapperImpl.getPropertyValue(BeanWrapperImpl.java:699)
              	at org.springframework.batch.core.scope.StepScope.resolveContextualObject(StepScope.java:131)
              	at org.springframework.beans.factory.config.BeanExpressionContext.containsObject(BeanExpressionContext.java:51)
              	at org.springframework.context.expression.BeanExpressionContextAccessor.canRead(BeanExpressionContextAccessor.java:36)
              	at org.springframework.expression.spel.ast.PropertyOrFieldReference.readProperty(PropertyOrFieldReference.java:189)
              	at org.springframework.expression.spel.ast.PropertyOrFieldReference.getValueInternal(PropertyOrFieldReference.java:71)
              	at org.springframework.expression.spel.ast.CompoundExpression.getValueInternal(CompoundExpression.java:52)
              	at org.springframework.expression.spel.ast.Indexer.getValueInternal(Indexer.java:83)
              	at org.springframework.expression.spel.ast.CompoundExpression.getValueInternal(CompoundExpression.java:57)
              	at org.springframework.expression.spel.ast.SpelNodeImpl.getValue(SpelNodeImpl.java:93)
              	at org.springframework.expression.spel.standard.SpelExpression.getValue(SpelExpression.java:88)
              	at org.springframework.context.expression.StandardBeanExpressionResolver.evaluate(StandardBeanExpressionResolver.java:138)
              	... 26 more

              Comment


              • #8
                You just have to escape the property name if it has a period in it (because that's a special symbol in SpEL), e.g. #{jobParameters['output.file']}. Read the Spring user guide for more information about SpEL (it's not a Spring Batch feature).

                Comment


                • #9
                  Thank you Dave. It works!!!

                  I will read the SpEL documentation.

                  Comment


                  • #10
                    Small correction

                    Dave's example is close but there is a typo (I found it when sending someone else to this thread). The following is correct (move one of the close parens to after System instead of after currentTimeMillis()):

                    Code:
                    <bean id="writer" class="...FlatFileItemWriter">
                       <property name="resource" value="file://.../myfile-#{T (java.lang.System).currentTimeMillis()}.txt"/>
                        ...
                    </bean>

                    Comment


                    • #11
                      I had the exact same requirement. I used Spel to get the params from job params so my file name is dynamic. But I like to keep the file in *.tmp state, because I do not want other processes to read the file while I am writing. So I added a final step after all the writing is done , I am adding a datetimestamp and a .xml(say) using new file().renameTo (new name). This makes sure that the file has been written successfully so is ready to be picked up/read.

                      Comment

                      Working...
                      X