Announcement Announcement Module
Collapse
No announcement yet.
Summary and running totals Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • Summary and running totals

    Where and how would be the best way of keeping running total etc for all the processed items. (I need to write a control record at the end of a fixed length file, containing total number of record etc).

    The same question applies for my summary report, which is a summary of all data processed and extracted to flat file.

  • #2
    You should have a "total" variable on your writer. Then use an AfterWrite method (again, on your writer) to add the item information to the total.

    Code:
    @AfterWrite
    public void updateTotalPrice(List<Trade> trades) {
        for (Trade trade : trades) {
            this.totalPrice = this.totalPrice.add(trade.getPrice());
        }
    }
    In the given example, it might also be a good idea to assert that trade.getPrice() is not null in the write() method. This will ensure that a trade without a price is skipped since a NullPointerException in an AfterWrite will not properly skip.

    To make sure that this data is available on restart, you need to make your writer implements ItemStream so that you can use the open() and update() methods to keep the total up to date:

    Code:
    public void open(ExecutionContext executionContext) throws ItemStreamException {
        if (executionContext.containsKey(TOTAL_AMOUNT_KEY)) {
            this.totalPrice = (BigDecimal) executionContext.get(TOTAL_AMOUNT_KEY);
        }
        else {
            // Fresh run. Disregard old state.
            this.totalPrice = BigDecimal.ZERO;
        }
    }
    
    public void update(ExecutionContext executionContext) {
        executionContext.put(TOTAL_AMOUNT_KEY, this.totalPrice);
    }
    To store this information to write out in a later step, you might want to use the Job's ExecutionContext. In an AfterStep method you can promote items from the Step's ExecutionContext to the Job's. This needs to be done in the AfterStep because the Job's ExecutionContext is only persisted at the end of the step. Spring Batch actually contains a component to do just this: the ExecutionContextPromotionListener.
    Last edited by DHGarrette; Mar 6th, 2009, 08:58 AM.

    Comment


    • #3
      Thanks,

      This will help me to keep track of all the totals, as well as persist my state in case of failure.

      1 question, where do I define the "TOTAL_AMOUNT_KEY" ?

      Comment


      • #4
        Originally posted by jaenswart View Post
        1 question, where do I define the "TOTAL_AMOUNT_KEY" ?
        In your ItemWriter?

        Comment


        • #5
          Dave,

          If the TOTAL_AMOUNT_KEY is defined within the ItemWriter on 1 step (and within that context and for restart semantics I understand that), how would you reference that same final from a next step (Let's say a summarization step) in order to get it from the executionContext?

          executionContext.get("TOTAL_AMOUNT_KEY"), now "TOTAL_AMOUNT_KEY" is undefined for any class in the summarization step?

          Basically, how would any subsequent code that wants to use that value, pull it from the Map, if it doesn't know the key?

          Comment


          • #6
            I would just use a public static field in the writer. You could externalise it somewhere if that makes you feel more comfortable.

            Comment


            • #7
              100% thanks

              Comment


              • #8
                I am trying to calculate the summary of total as follows ..


                Writer class

                public void write( List<?extends Customer> list )
                throws Exception
                {
                logger.logExecutingMethod( );
                logger.debug( "Size :::" + list.size( ) );

                int size = ( ( list != null ) && ( list.size( ) > 0 ) ) ? list.size( ) : 0;

                delegates.write( list );
                totalRows += size;
                logger.debug( "Total Rows :::" + totalRows );
                logger.logExitingMethod( );
                }

                /* (non-Javadoc)
                * @see org.springframework.batch.item.file.FlatFileFooter Callback#writeFooter(java.io.Writer)
                */
                public void writeFooter( Writer writer )
                throws IOException
                {
                logger.logExecutingMethod( );

                logger.debug( "Total Rows :::writeFooter " + totalRows );
                writer.write( "Total Rows Processed: " + totalRows );
                logger.logExitingMethod( );
                }


                context xml

                <bean id="flatFileWriter" class="com.core.project.adhoc.writer.AdhocReportWr iter">
                <property name="delegates" ref="fileWriter"></property>

                </bean>

                <bean id="fileWriter" class="org.springframework.batch.item.file.FlatFil eItemWriter"
                p:resource="file:${user.home}/csv/customerList.csv">

                <property name="lineAggregator">
                <bean id="lineAggregator"
                class="org.springframework.batch.item.file.transfo rm.DelimitedLineAggregator">
                <property name="delimiter">
                <value>,</value>
                </property>

                <property name="fieldExtractor">
                <bean
                class="org.springframework.batch.item.file.transfo rm.BeanWrapperFieldExtractor">
                <property name="names">
                <value> BillingId,
                CpAccountId,ProductCode,ProductPicCode,StartDate,E ndDate,ProductDescription
                </value>
                </property>
                </bean>
                </property>
                </bean>
                </property>
                <property name="headerCallback" ref="adhocReportHeader"></property>
                <property name="footerCallback" ref="flatFileWriter"></property>
                </bean>


                Please advice me ..........



                Param

                Comment


                • #9
                  Fixed ... please try the above with following



                  <bean id="flatFileWriter" class="com.core.project.adhoc.writer.AdhocReportWr iter">
                  <property name="delegates" ref="fieldExtractor"></property>

                  </bean>

                  <bean id="fileWriter" class="org.springframework.batch.item.file.FlatFil eItemWriter"
                  p:resource="file:${user.home}/csv/customerList.csv">

                  <property name="lineAggregator">
                  <bean id="lineAggregator"
                  class="org.springframework.batch.item.file.transfo rm.DelimitedLineAggregator">
                  <property name="delimiter">
                  <value>,</value>
                  </property>

                  <property name="fieldExtractor">
                  <bean
                  class="org.springframework.batch.item.file.transfo rm.BeanWrapperFieldExtractor">
                  <property name="names">
                  <value> BillingId,
                  CpAccountId,ProductCode,ProductPicCode,StartDate,E ndDate,ProductDescription
                  </value>
                  </property>
                  </bean>
                  </property>
                  </bean>
                  </property>
                  <property name="headerCallback" ref="adhocReportHeader"></property>
                  <property name="footerCallback" ref="flatFileWriter"></property>
                  </bean>



                  Instead inject the Itemwriter interface in our writer use FlatFileItemWriter .. ... Its working

                  Comment

                  Working...
                  X