We have a job configured like this :
Batch configuration :
AddAddressToPersonWriter class :
We launch the batch with a test which initialize 15 persons in the @BeforeClass.
15 persons are read by JpaPagingItemReader and modified by our AddAddressToPersonWriter but at the end, we can see 12 addresses in the database instead of 15.
Our commit-interval is 4, so 12 commited modifications = 3 pages * 4 items per page, and we have the 3 last records not modified.
Output logs :
We noticed that the JpaPagingItemReader has been modified since version 1.1.3 : the flush has been moved from the end of the method doReadPage to the beginning of this method. If we write our own JpaPagingItemReader with a flush at the end of the doReadPage method, we solve our problem.
So :
- why was the flush moved from the end to the beginning of the method ?
- is there any risk in replacing it at the end ?
- is there a Jira that references this pb and will it be soon fixed ?
Thank you for your answers,
JP
Batch configuration :
Code:
<batch:job id="minimal" job-repository="jobRepository" restartable="true"> <batch:step id="step1"> <batch:tasklet> <batch:chunk reader="jpaPersonReader" writer="addAddressToPersonWriter" commit-interval="4" /> </batch:tasklet> </batch:step> </batch:job> <bean id="jpaPersonReader" class="org.springframework.batch.item.database.JpaPagingItemReader"> <property name="entityManagerFactory" ref="entityManagerFactory" /> <property name="queryString" value="select p from Person p order by p.id" /> <property name="pageSize" value="4" /> </bean> <bean id="addAddressToPersonWriter" class="org.jp.spring_batch_labs.AddAddressToPersonWriter" />
Code:
public class AddAddressToPersonWriter implements ItemWriter<Person> { private static final Logger LOG = Logger.getLogger(AddAddressToPersonWriter.class); @Override public void write(List<? extends Person> items) throws Exception { LOG.info("updating " + items.size() + " persons."); for (Person person : items) { Address address = new Address(); address.setCp(Long.toString(System.currentTimeMillis())); person.addAddress(address); } } }
15 persons are read by JpaPagingItemReader and modified by our AddAddressToPersonWriter but at the end, we can see 12 addresses in the database instead of 15.
Our commit-interval is 4, so 12 commited modifications = 3 pages * 4 items per page, and we have the 3 last records not modified.
Output logs :
Code:
-- INFO org.springframework.batch.core.launch.support.SimpleJobLauncher - Job: [FlowJob: [name=minimal]] launched with the following parameters: [{param1=1}] -- INFO org.springframework.batch.core.job.SimpleStepHandler - Executing step: [TaskletStep: [name=step1]] -- DEBUG org.springframework.batch.item.database.JpaPagingItemReader - Reading page 0 -- INFO org.jp.spring_batch_labs.AddAddressToPersonWriter - updating 4 persons. -- DEBUG org.springframework.batch.item.database.JpaPagingItemReader - Reading page 1 -- INFO org.jp.spring_batch_labs.AddAddressToPersonWriter - updating 4 persons. -- DEBUG org.springframework.batch.item.database.JpaPagingItemReader - Reading page 2 -- INFO org.jp.spring_batch_labs.AddAddressToPersonWriter - updating 4 persons. -- DEBUG org.springframework.batch.item.database.JpaPagingItemReader - Reading page 3 -- INFO org.jp.spring_batch_labs.AddAddressToPersonWriter - updating 3 persons. -- INFO org.springframework.batch.core.launch.support.SimpleJobLauncher - Job: [FlowJob: [name=minimal]] completed with the following parameters: [{param1=1}] and the following status: [COMPLETED] -- INFO org.jp.spring_batch_labs.SimpleTestJpa - 12 adresses in the db
We noticed that the JpaPagingItemReader has been modified since version 1.1.3 : the flush has been moved from the end of the method doReadPage to the beginning of this method. If we write our own JpaPagingItemReader with a flush at the end of the doReadPage method, we solve our problem.
So :
- why was the flush moved from the end to the beginning of the method ?
- is there any risk in replacing it at the end ?
- is there a Jira that references this pb and will it be soon fixed ?
Thank you for your answers,
JP
Comment