Announcement Announcement Module
Collapse
No announcement yet.
How to stop file transferring in spring-batch Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • How to stop file transferring in spring-batch

    I have created a spring-batch job for reading files from local directory and upload it to remote directory through ftp using Camel-spring-batch. I am doing the same using chunk. My spring batch job configuration looks like :

    Code:
    <bean id="consumerTemplate" class="org.apache.camel.impl.DefaultConsumerTemplate" init-method="start" destroy-method="stop">
        <constructor-arg ref="camelContext"/>
    </bean>
    
    <bean id="producerTemplate" class="org.apache.camel.impl.DefaultProducerTemplate" scope="step" init-method="start" destroy-method="stop">
        <constructor-arg ref="camelContext"/>
    </bean>
    
    <bean id="localFileReader" class="com.camel.springbatch.reader.LocalFileReader" scope="step" destroy-method="stop">
        <constructor-arg value="file:#{jobParameters['dirPath']}"/>
        <constructor-arg ref="consumerTemplate"/>
    </bean>
    
    <bean id="ftpFileWriter" class="com.camel.springbatch.writer.FtpFileWriter" scope="step">
        <constructor-arg ref="producerTemplate"/>
        <constructor-arg value="ftp://#{jobParameters['host']}?username=#{jobParameters['user']}&amp;password=#{jobParameters['password']}"/>
    </bean>
    Job configuration :
    Code:
    <batch:job id="ftpReadWrite">
        <batch:step id="readFromLocalWriteToFtp" next="readFromFtpWriteToLocal">
            <batch:tasklet>
                <batch:chunk reader="localFileReader" writer="ftpFileWriter"  commit-interval="5" />
            </batch:tasklet>
        </batch:step>
    And my "Localfilereader" and "ftpFileWriter" looks like :

    Code:
    import org.apache.camel.ConsumerTemplate;
    import org.apache.camel.component.spring.batch.support.CamelItemReader;
    import org.slf4j.Logger;
    import org.slf4j.LoggerFactory;
    
    public class LocalFileReader extends CamelItemReader {
    private Logger log= LoggerFactory.getLogger(this.getClass());
    ConsumerTemplate consumerTemplate;
    String endpointUri;
    
    public LocalFileReader(ConsumerTemplate consumerTemplate, String endpointUri) {
        super(consumerTemplate, endpointUri);
        this.consumerTemplate=consumerTemplate;
        this.endpointUri=endpointUri;
    }
    
    @Override
    public Object read() throws Exception {
        Object item = consumerTemplate.receiveBody(endpointUri);
        return item;
    }}
    "Ftp File Writer"
    Code:
    import org.apache.camel.ProducerTemplate;
    import org.apache.camel.component.spring.batch.support.CamelItemWriter;
    import org.slf4j.Logger;
    import org.slf4j.LoggerFactory;
    import java.util.List;
    public class FtpFileWriter extends CamelItemWriter {
    private Logger log= LoggerFactory.getLogger(this.getClass());
    ProducerTemplate producerTemplate;
    String endpointUri;
    public FtpFileWriter(ProducerTemplate producerTemplate, String endpointUri) {
        super(producerTemplate, endpointUri);
        this.producerTemplate=producerTemplate;
        this.endpointUri=endpointUri;
    }
    
    @Override
    public void write(List items) throws Exception {
        System.out.println("************************Writing item to ftp "+items);
        for (Object item : items) {
            System.out.println("writing item [{}]..."+item);
            producerTemplate.sendBody(endpointUri, item);
            log.debug("wrote item");
        }
    }
    }
    It works fine if I have only 5 file in my local directory. It read the all 5 file from my local directory and it send to the writer and writer send it to the ftp server as my commit-interval=5. If I have 6 file in in local directory then it send first chunk of 5 file to writer and again it start reading the remaining file and this time there is only one file remaining. It read 1 file and start waiting for 4 files and never send to writer. I tried it with commit-interval=1 now it send all 6 files to server and again start waiting for next file. Here I need to stop the process once all file have been processed.

    Please help me to resolved this issue...

  • #2
    Is there a reason you aren't using Spring Integration for this? It would be much easier IMHO and take this type of functionality out of your batch job.

    Comment

    Working...
    X