Announcement Announcement Module
Collapse
No announcement yet.
Introduce FieldSetMetaData and FieldMetaData classes to the infrastructure package Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • Introduce FieldSetMetaData and FieldMetaData classes to the infrastructure package

    Introduce some meta-data classes to the infrastructure package. Specifically a FieldSetMetaData and FieldMetaData class. The FieldMetaData would look similar to a ResultSetMetaData or RowSetMetaData but would have some additional properties added (i.e. PropertyEditor(s), alignment, padding, pattern, etc).

    There are a number of reasons that I have for introducing those classes:
    • From the Spring Batch documention,
      a FieldSet is conceptually very similar to a Jdbc ResultSetMetaData or RowSetMetaData
      Both of those classes however, offer a way to get at the metadata object for the row, via getMetaData().
    • DefaultFieldSet already contains some "meta-data", though currently that is only the field names.
    • Companies I've worked at have implemented solutions that used metadata at the field and row level. Commercial ETL tools I'm familiar with also implement some sort of metadata at the field/row level.
    • DefaultFieldSet currently is limited in the way it can parse tokens (see 50182 for an example). Having metadata at the field level allows users to set their own PropertyEditors .
    • Sometime it's overkill to map a row into a domain object (i.e. when you're just displaying the rows of a file on a screen or directly dumping them into a database), or rows don't map easily into domain objects for a file. See 49347
    • DefaultFieldSet is not consistent with overloads to methods regarding patterns and default values. Can't all objects have defaults and patterns applied?
    • Padding - can't it be applied to both delimited and fixed length tokenizers? What about padding when outputting?
    • Offer an "allow null" for fields containing an empty string.
    • Offer users the option of returning trimmed Strings or untrimmed Strings. This was discussed here:
      48519, and fixed here: BATCH-289. I believe the 2 classes I'm proposing would offer an alternative approach to this.
    • It would help with this issue: BATCH-261
    • It would help with some of these issues 47861 and with this issue 50182
    • Would allow a FieldSet "type validator" to be introduced.
    • When you're importing a text file in Excel, it allows you to set metadata like type and format for each column/field.

    Future Offerrings Might include:
    • Introducing an adapter class for the JSTL ResultSupport and ResultImpl classes that know how to display data columns on a JSP via their meta-data.
    • Introducing a class that maps a FieldSet to a database row (maybe via the BeanWrapperFieldSetMapper) merely using the FieldSetMetaData. See 49347.
    • Spring namespace support for setting the meta-data info describing the FieldSets.
    • Some PropertyEditors that batch users might typically use like a CobolBigDecimalEditor, see 50182, that could then be set on the FieldMetaData.

    Initially I see the AbstractLineTokenizer and DefaultFieldSet making the most use of these classes but a lot of possibilities open up if they were introduced.

    Here's some preliminary code for implementing these metadata classes and a FieldSet "type validator": fieldset-metadata.zip. The posting tool limits the size of image files so I've added the class diagram as a separate zip: FieldSetMetaData-class-diagram.zip
    I'll open a JIRA issue for this as well.


    Using the code I've attached here are some basic samples of different combinations of what can be done within the applicationContext files (and as I've noted this can be improved upon later with Spring namespaces):
    Code:
    <bean id="headerRecordDescriptor" class="org.springframework.batch.io.file.transform.DelimitedLineTokenizer">
    	<property name="fields">		
    		<bean class="org.springframework.batch.io.file.metadata.FieldSetMetaDataImpl" >			
    			<property name="names"    value="LINE_ID,ORDER_ID  ,ORDER_DATE"/>
    			
    			<!-- notice if type is empty string it defaults to type String  -->
    			<property name="types"    value="       ,long      ,java.util.Date" />
    			
    			<!-- notice if pattern is empty it defaults to null  -->
    			<property name="patterns" value="       ,##########" />
    		</bean>
    	</property>			
    </bean>
    
    
    <bean id="itemLineDescriptor" class="org.springframework.batch.io.file.transform.DelimitedLineTokenizer">	
    	<property name="fields">
    		<bean class="org.springframework.batch.io.file.metadata.FieldSetMetaDataImpl" >	
    			<property name="names"         value="LINE_ID,ITEM_ID,PRICE,               DISCOUNT_PERC,       DISCOUNT_AMOUNT,     SHIPPING_PRICE,      HANDLING_PRICE,      QUANTITY,TOTAL_PRICE" />
    			<property name="types"         value="       ,long,   java.math.BigDecimal,java.math.BigDecimal,java.math.BigDecimal,java.math.BigDecimal,java.math.BigDecimal,int,     java.math.BigDecimal" />
    			<property name="patterns"      value="                ,########.##" />
    			
    			<!-- notice defaultValues as Strings getting converted to proper requiredType -->
    			<property name="defaultValues" value="                ,                   ,                    ,                    ,9.99                ,1.99" />
    			
    			<!-- example of defaultValues as Objects -->
    			<property name="fields[7].defaultValue">
    				<value type="int">1</value>									
    			</property>
    		</bean>
    	</property>			
    </bean>
    
    
    <bean id="footerRecordDescriptor" class="org.springframework.batch.io.file.transform.DelimitedLineTokenizer">	
    	<property name="fields" value="LINE_ID,         TOTAL_LINE_ITEMS,TOTAL_ITEMS,TOTAL_PRICE" />
    	<property name="types"  value="java.lang.String,int,             int,        java.math.BigDecimal" />
    </bean>

  • #2
    JIRA issue here: BATCH-441

    Comment

    Working...
    X