Announcement Announcement Module
Collapse
No announcement yet.
Db input, CSV text file output... Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • Db input, CSV text file output...

    Our batch processing needs are pretty simple: wake up at 12:01 noon each Tuesday morning and run a job which will read a Database table - retrieving specific rows, do some transformation and output a comma delimited CSV file, then email the file to some recipients as an attachement...

    Questions:
    1) Which of the examples that ships with spring-batch.1.0.0.m3 might be useful to explore for this style of batch?

    2) Would like to use crontab on Unix to schedule the job to run - could someone show a simple example on how to setup a classpath in crontab that might work?

    3) I couldn't find it in the documentation but I'm assuming that I can package a spring-batch application as a stand-alone JAR and deploy it on any server... provided that this server has connectivity to the database resources the appliction needs to access?

    thanks in advance -
    James

  • #2
    1) Which of the examples that ships with spring-batch.1.0.0.m3 might be useful to explore for this style of batch?
    3rd step of tradeJob should be a perfect match. There is also a job matrix document describing what features each sample demonstrates (see BATCH-247 for a reasonably up-to-date version).

    2) Would like to use crontab on Unix to schedule the job to run - could someone show a simple example on how to setup a classpath in crontab that might work?
    see http://forum.springframework.org/sho...t=48352&page=2

    3) I couldn't find it in the documentation but I'm assuming that I can package a spring-batch application as a stand-alone JAR and deploy it on any server... provided that this server has connectivity to the database resources the appliction needs to access?
    yes

    Comment


    • #3
      Sending e-mail as final step...

      Great stuff - thanks. That should do the trick for the first part of this job.

      This job needs to execute in several steps: extract data, format output, send email with generated report as an attachement. Are multiple job steps possible here?

      Finally, I understand that spring-batch does not have email support (yet?), but couldn't we make use of Spring's JavaMailSender utility to send an email with attachement?

      Comment


      • #4
        This job needs to execute in several steps: extract data, format output, send email with generated report as an attachement. Are multiple job steps possible here?
        If you want to execute all "steps" for one record and then move on to another record then you need to put all "steps" into a single Spring-Batch Step. If you want to apply first "step" to all records and move on to second "step" to all records then you can map the "steps" to Spring-Batch Steps. (Hope it is not confusing).

        Finally, I understand that spring-batch does not have email support (yet?), but couldn't we make use of Spring's JavaMailSender utility to send an email with attachement?
        There is no email support provided out-of-box, but using Spring's MailSender should be straightforward, see http://forum.springframework.org/showthread.php?t=48530

        Comment


        • #5
          Multiple inputs into a step?

          I've been analyzing the tradeJob sample - think it will be a good template for my needs. Have some questions:

          TX doesn't matter too much to me in this case - it's just extract from db, format, and email. My job run needs to be structured something like this:

          Step1
          o Input: SQL query to get the data, pass to the processor
          o Processor: format all data from sql query into CSV, write out to local file system (i.e. /tmp/output/2000122_CUSTOMERREPORT.TXT) as CSV output file.

          Step2
          o Input 1: SQL query to get the list of recipients that want the report (from step 1)
          o Input 2 (possible?): filename of the report from completion of Step 1 (i.e. /tmp/output/2000122_CUSTOMERREPORT.TXT)
          o Processor: use subscriber names/emails from Input1 and formulate attachement (from Input2) to send out email to the recipients including the attached report (from Step1)

          I'm not understanding how to provide TWO inputs into the second step: 1) List of recipients from DB, AND 2) Fully qualified file path to the report from step2... Maybe there is some way of sharing data between steps? Like: results of Step1 are handed over as input into Step 2 somehow? Please clarify...

          I've been looking for JavaDocs or a Reference manual on Spring-batch but can't seem to find it, so a bit of help here would be appreciated as well.

          Thanks in advance -
          James
          Last edited by jdepaul; Jan 22nd, 2008, 04:03 PM.

          Comment


          • #6
            JavaDocs and reference documentation can be found here:

            http://static.springframework.org/spring-batch/

            The reference documentation still needs some work, but the java docs should be good. You should also be able to view the source code itself by attaching through maven or downloading directly from the zip distribution.

            I'm not sure I fully understand your question about having two inputs? You can wire anything you like in spring, so you could easily wire in two ItemReaders.

            Comment


            • #7
              This article might be helpful as well:

              http://www.theserverside.com/tt/arti...gBatchOverview

              Comment


              • #8
                o Input 2 (possible?): filename of the report from completion of Step 1 (i.e. /tmp/output/2000122_CUSTOMERREPORT.TXT)
                o Processor: use subscriber names/emails from Input1 and formulate attachement (from Input2) to send out email to the recipients including the attached report (from Step1)
                If I understand correctly you'll be sending the same file as attachment to all recipients, in that case you don't really need two inputs - the file name is more a parameter of the mail-sending processor. You can use the same property placeholder for output file in the first step and (let's say) 'attachment' property of the processor used in the second step.

                If the above wasn't sufficient I remember adding something like 'JobScope' had been considered in the past, but there was no usecase for it so it was put aside until proved useful.

                Comment

                Working...
                X