Announcement Announcement Module
Collapse
No announcement yet.
Passing an object between Listener and Writer Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • #16
    Originally posted by dkaminsky View Post
    I'm confused as to what you are trying to do - is this in relation to the same issue that I just responded to? (if it is, you lost me)
    Sorry for the confusion. This is with respect to your last reponse. I am using ABCMapper as a fieldSetMapper. So i thought that's what (ABCMapper) you meant by "item reader". (am new to Spring and Spring Batch..) What do you mean by "item reader"?

    Comment


    • #17
      Chapter 3 in the reference documentation covers ItemReaders and ItemWriters in great detail:

      http://static.springframework.org/sp...structure.html

      Comment


      • #18
        Take a look at what Lucas posted, but to clarify, by the item reader I mean the FlatFileItemReader -- when I say add to / modify the item reader, I mean to subclass it (or copy and paste into your own class) to add custom logic. e.g.

        Code:
        public class InputCapturingFlatFileItemReader extends FlatFileItemReader {
          private Object lastRead = null;
        
          public Object read() throws Exception {
            // duplicate logic of parent class but capture raw data in lastRead even if exception is thrown
          }
        
          public void itemSkipCallback() {
            // put your code here to evaluate lastRead and persist to database
            // call this method from onSkipInRead by injecting the instantiated bean 
            // of this type into your skip listener implementation
          }
        }

        Comment


        • #19
          (Like I said, this is a work-around until the issue is officially addressed)

          Comment


          • #20
            I might be missing something, as this thread has gotten quite long, but from looking at your code above Doug it looks like you (and I'm assuming SpringForever) are trying to get access to the original input. Keep in mind that when there's a parsing exception in the FlatFileItemReader, a FlatFileParsingException is thrown, which contains the original input line:

            Code:
            	public Object read() throws Exception {
            		String line = readLine();
            
            		if (line != null) {
            			try {
            				FieldSet tokenizedLine = tokenizer.tokenize(line);
            				return fieldSetMapper.mapLine(tokenizedLine);
            			} catch (RuntimeException ex) {
            				// add current line count to message and re-throw
            				int lineCount = getReader().getPosition();
            				throw new FlatFileParseException("Parsing error at line: " + lineCount + " in resource=" + path
            				        + ", input=[" + line + "]", ex, line, lineCount);
            			}
            		}
            		return null;
            	}
            If you were coding an ItemReaderListner, all you would need to do is a simple instanceof check for the FlatFileParseExceptin, from which you could call getInput().

            Comment


            • #21
              You understand correctly - it's related to BATCH-540 - he wants to retrieve the raw input data if there's an exception during read - case in point, if tokenization fails.

              A more permanent solution would be to pass this in as an argument to onSkipInRead, just like onSkipInWrite, except with the potential that it could be equal to null (e.g. if the exception is because the file can't be opened, etc.).

              Here's the alternative, springforever:

              Code:
              String originalInput = exception.getInput();
              This only solves for the FlatFileItemReader though, and not for other item reader implementations since their exceptions don't necessarily house extra state.

              I don't think of this normally since people seem to cringe whenever I suggest making the state of an exception meaningful, but since we're laying out all the options...
              Last edited by dkaminsky; Apr 8th, 2008, 04:46 PM. Reason: zounds, foiled again

              Comment


              • #22
                I'm not sure it's really the 'state of an exception' It's a parsing exception, and it contains the original input that caused the exception, which is pretty common. We already log this out in other ways, in some cases as a stack trace which contains it. I've had a client create a specific table that contained generic stack traces that could be written to, in the case of the FlatFileParseException, you would have the original line. There isn't too much you can do for database input though. It might be a good idea to package up the original exception with the row we were on in that case. At least then you could run the query and pull back the record with that row number from the set to figure out why the job failed.

                Comment


                • #23
                  I guess it isn't stateful, strictly speaking, since it's immutable - I think I might be projecting my frustrations with other situations onto this one...

                  So, yes, it seems I had forgotten that FlatFileParseException had that special property, so I apologize for any confusion I caused.

                  Comment


                  • #24
                    Thank you all...... Few concepts are more clear to me now... Keep up the good work!
                    Last edited by springforever; Apr 9th, 2008, 08:30 AM.

                    Comment


                    • #25
                      Originally posted by dkaminsky View Post
                      1. Have your writer implement ItemStream (or extend AbstractItemStreamItemWriter if you are not already extending another class) if it doesn't already do so.

                      2. Add a private ExecutionContext member variable

                      3. Implement/Override/Add To the "open(ExecutionContext)" method to set the member execution context to the passed execution context. (If you are not extending the Abstract implementation, you may also need to add a no-op implementation of update(ExecutionContext) and close(ExecutionContext).

                      4. Add to / retrieve from the execution context whenever you want during the write method.

                      //QED

                      This is currently the best practice for making ItemReader/Writer collaborators "ExecutionContext aware"

                      Hope this helps
                      Hi,

                      Question 1
                      Step-1....SkipLimitStepFactoryBean....blank writer.... listener (putting loadId in stepExecutionContext)

                      Step-2....SimpleStepFactoryBean......FileWriter extends AbstractItemStreamItemWriter.....has open method...

                      I am getting ExecutionContext in FileWriter. However, there is no key "loadId"... This is key is there in the database. When i attach stpe-2 with listener then it works fine.... Why is that? Can i not get this key in step-2 without attaching it to the listener?

                      Questions-2
                      How do i actually terminate a job after step-1 , if there is flag set in the afterStep of step-1 (flag is for termination...i am using stepExecution.setTerminateOnly()...).... still job continues with step-2...

                      Comment


                      • #26
                        1. ExecutionContext is created as new for each step - the best practice to pass data (eg. an execution context) between steps is to store it in a listener member variable and assigning the same listener to both steps. This can be done by simply assigning the data in one step (in afterStep) and restoring it in the next step (in beforeStep).

                        2. Throw an exception that is non-recoverable (i.e. not skippable, doesn't cause a retry, etc.)

                        Comment


                        • #27
                          Originally posted by dkaminsky View Post
                          1. ExecutionContext is created as new for each step - the best practice to pass data (eg. an execution context) between steps is to store it in a listener member variable and assigning the same listener to both steps. This can be done by simply assigning the data in one step (in afterStep) and restoring it in the next step (in beforeStep).

                          2. Throw an exception that is non-recoverable (i.e. not skippable, doesn't cause a retry, etc.)
                          Response 1, above works (Using separate listeners for step-1 and step-2). Thank you! Now, i am trying to access the FileWriter above from the CompositeWriter. executionContext in the CompositeWriter gets loadId. But, FileWriter shows "null" as executionContext..... Do i need to set executionContext of FileWriter from CompositeWriter? Just wanted to make sure if that would be the write approach....

                          Comment


                          • #28
                            Not sure i understand your question

                            Comment


                            • #29
                              Did you forget to register the wrapped writer as an ItemStream in the step?

                              Comment


                              • #30
                                1. CompositeItemWriter includes FileItemWriter and ItemWriter_2.
                                2. CompositeItemWriter also has a variable tempItemWriter
                                3. If some condition is true then tempItemWriter=FileItemWriter
                                4. tempItemWriter.write(item) will be called in CompositeItemWriter.

                                A. executionContext in the CompositeItemWriter gets the key loadId.
                                B. executionContext in the FileItemWriter is null. (Need to get loadId here and executionContext should not be null)

                                -------------Config-------
                                <bean id="someFileWriter" class="CompositeItemWriter">
                                <constructor-arg>
                                <bean class="FileItemWriter">
                                <property name="fileDao">
                                <bean class="fileDaoImpl"/>
                                </property>
                                </bean>
                                </constructor-arg>
                                <constructor-arg>
                                ....ItemWriter_2
                                </constructor-arg>
                                </bean>
                                ----------

                                Please let me know if it's still not clear and i will post a sample code....

                                Comment

                                Working...
                                X