Announcement Announcement Module
Collapse
No announcement yet.
What is your persistance choice for Spring Batch Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • What is your persistance choice for Spring Batch

    Hi all,

    I am still a newbie to Spring Batch. When it comes to persisting data, I see that SB supports JDBC, Hiberate, JPA, iBatis and possibly others. I know there are strenghts and drawbacks to each of these, but I was wondering if you had a prefrence and why? I have heard JDBC is still best for performance reasons, is this true? Any guidance would be appreciated.

    Thanks.

  • #2
    it really depends on the usecase and other circumstances

    for simple read/write batches and pure performance i would go with
    • jdbc
    • tailored and checked sql (checked from senior dba) which uses database dialect features
    • JdbcTemplate.batchUpdate(...) (*)
    • with BatchPreparedStatementSetter

    Why not JdbcBatchItemWriter(with ItemPreparedStatementSetter or ItemSqlParameterSourceProvider)
    /SimpleJdbcInsertOperations
    /SimpleJdbcTemplate?

    performance, at least in my tests its quite slow compared to a pure jdbcTemplate

    My Ranking so far:
    1. JdbcTemplate/raw Jdbc
    2. SimpleJdbcTemplate with SqlParameterSourceUtils.createBatch(...);
    3. JdbcBatchItemWriter with ItemSqlParameterSourceProvider
    4. JdbcBatchItemWriter with ItemPreparedStatementSetter
    5. SimpleJdbcInsertOperations with SqlParameterSourceUtils.createBatch(...);

    The order changes a bit with available CPU (think 2 Cores+), but the Nr.1 JdbcTemplate/raw jdbc stays there.


    Why not JPA ? Well JPA can be quite fast too, but you always loose some time for the mapping inside the ORM tool and the work to get the JPA solution there is just more than with JDBC. But this depends on the circumstances too, in some companies does not exist any database know how, but its good for e.g. Hibernate, TopLink etc.

    This very usecase can be made even faster, if you use something like
    • no batch
    • tailored and checked sql
    • temp table created within the sql, which can be used for the data transformations, think of sql code over more than 2 pages :-)
    • JCL (Z/OS Host) or some other simple "run sql" script which runs the "delete all from target table, copy all from temp-target to target" sql

    you would miss some nice features like restart with this, but it is fast

    (*) or raw jdbc, but my tests show no significant difference between JdbcTemplate and raw jdbc
    Last edited by michael.lange; Oct 20th, 2011, 03:37 PM.

    Comment

    Working...
    X