Announcement Announcement Module
Collapse
No announcement yet.
FlatfileItemReader failed to initialize while reading translated language characters Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • FlatfileItemReader failed to initialize while reading translated language characters

    Hi

    I am getting a problem while reading a csv file from FlatFileItemReader in spring batch . It gives me a exception only when i am having the characters like ,, (means the translated languages characters ) in my file . Without these characters i m able to read the file from FTP server . Also this problem doesnt come when i am using my Tomcat5.5 server from eclipse for reading the file . But if i try to run the same code deployed on my Websphere i.e WAS 6 \Linux , i face this problem . Below is my application context xml i am using

    <beans:bean id="reader" class="org.springframework.batch.item.file.FlatFil eItemReader" scope="step">
    <beansroperty name="resource" value="#{jobParameters[ftp.file.name]}" />
    ...

    I have also tried by giving the property name as encoding and specified as UTF-8 , but still it didnt worked .
    <beansroperty name="encoding" value="UTF-8"/>


    Below is the exception i get :

    org.springframework.batch.item.ItemStreamException : Failed to initialize the reader
    at org.springframework.batch.item.support.AbstractIte mCountingItemStreamItemReader.open(AbstractItemCou ntingItemStreamItemReader.java:137)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Nativ e Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(Native MethodAccessorImpl.java:79)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(De legatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:618)
    at org.springframework.aop.support.AopUtils.invokeJoi npointUsingReflection(AopUtils.java:310)
    at org.springframework.aop.framework.ReflectiveMethod Invocation.invokeJoinpoint(ReflectiveMethodInvocat ion.java:182)
    at org.springframework.aop.framework.ReflectiveMethod Invocation.proceed(ReflectiveMethodInvocation.java :149)
    at org.springframework.aop.support.DelegatingIntroduc tionInterceptor.doProceed(DelegatingIntroductionIn terceptor.java:131)
    at org.springframework.aop.support.DelegatingIntroduc tionInterceptor.invoke(DelegatingIntroductionInter ceptor.java:119)
    at org.springframework.aop.framework.ReflectiveMethod Invocation.proceed(ReflectiveMethodInvocation.java :171)
    at org.springframework.aop.framework.JdkDynamicAopPro xy.invoke(JdkDynamicAopProxy.java:204)
    at $Proxy11.open(Unknown Source)
    at org.springframework.batch.item.support.CompositeIt emStream.open(CompositeItemStream.java:98)
    at org.springframework.batch.core.step.item.ChunkMoni tor.open(ChunkMonitor.java:105)
    at org.springframework.batch.item.support.CompositeIt emStream.open(CompositeItemStream.java:98)
    at org.springframework.batch.core.step.tasklet.Taskle tStep.open(TaskletStep.java:288)
    at org.springframework.batch.core.step.AbstractStep.e xecute(AbstractStep.java:193)
    at org.springframework.batch.core.job.SimpleStepHandl er.handleStep(SimpleStepHandler.java:135)
    at org.springframework.batch.core.job.flow.JobFlowExe cutor.executeStep(JobFlowExecutor.java:61)
    at org.springframework.batch.core.job.flow.support.st ate.StepState.handle(StepState.java:60)
    at org.springframework.batch.core.job.flow.support.Si mpleFlow.resume(SimpleFlow.java:144)
    at org.springframework.batch.core.job.flow.support.Si mpleFlow.start(SimpleFlow.java:124)
    at org.springframework.batch.core.job.flow.FlowJob.do Execute(FlowJob.java:135)
    at org.springframework.batch.core.job.AbstractJob.exe cute(AbstractJob.java:281)
    at org.springframework.batch.core.launch.support.Simp leJobLauncher$1.run(SimpleJobLauncher.java:120)

    Please suggest something as i am not able to find out any solution for this .

    Thanks and regards
    Sachin

  • #2
    Your stack strace is incomplete, so its hard to see what happened. Also, if you could post the whole bean definition for the reader it might help.

    (Please use [code][/code] tags to post code and stack traces.)

    Comment


    • #3
      Hi
      The code for reader bean is :
      Code:
      <beans:bean id="reader" class="org.springframework.batch.item.file.FlatFileItemReader" scope="step">
      		<beans:property name="resource" value="#{jobParameters[ftp.file.name]}" />
      		<beans:property name="encoding" value="UTF-8"/>
      		<beans:property name="lineMapper">
      			<beans:bean class="org.springframework.batch.item.file.mapping.DefaultLineMapper">
      				<beans:property name="lineTokenizer">
      					<beans:bean class="org.springframework.batch.item.file.transform.DelimitedLineTokenizer">
      						<beans:property name="names" value="itemNo,itemType,itemName" />
      						<beans:property name="delimiter" ref="TAB_SEPARATOR" />
      					</beans:bean>
      				</beans:property>
      				<beans:property name="fieldSetMapper">
      					<beans:bean class="org.springframework.batch.item.file.mapping.BeanWrapperFieldSetMapper">
      						<beans:property name="prototypeBeanName" value="inputObject" />
      					</beans:bean>
      				</beans:property>
      			</beans:bean>
      		</beans:property>
      		<beans:property name="linesToSkip" value="1" />
      	</beans:bean>
      The complete error log is :

      AbstractStep E org.springframework.batch.core.step.AbstractStep execute Encountered an error executing the step
      org.springframework.batch.item.ItemStreamException : Failed to initialize the reader
      at org.springframework.batch.item.support.AbstractIte mCountingItemStreamItemReader.open(AbstractItemCou ntingItemStreamItemReader.java:137)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Nativ e Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(Native MethodAccessorImpl.java:79)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(De legatingMethodAccessorImpl.java:43)
      at java.lang.reflect.Method.invoke(Method.java:618)
      at org.springframework.aop.support.AopUtils.invokeJoi npointUsingReflection(AopUtils.java:310)
      at org.springframework.aop.framework.ReflectiveMethod Invocation.invokeJoinpoint(ReflectiveMethodInvocat ion.java:182)
      at org.springframework.aop.framework.ReflectiveMethod Invocation.proceed(ReflectiveMethodInvocation.java :149)
      at org.springframework.aop.support.DelegatingIntroduc tionInterceptor.doProceed(DelegatingIntroductionIn terceptor.java:131)
      at org.springframework.aop.support.DelegatingIntroduc tionInterceptor.invoke(DelegatingIntroductionInter ceptor.java:119)
      at org.springframework.aop.framework.ReflectiveMethod Invocation.proceed(ReflectiveMethodInvocation.java :171)
      at org.springframework.aop.framework.JdkDynamicAopPro xy.invoke(JdkDynamicAopProxy.java:204)
      at $Proxy12.open(Unknown Source)
      at org.springframework.batch.item.support.CompositeIt emStream.open(CompositeItemStream.java:98)
      at org.springframework.batch.core.step.item.ChunkMoni tor.open(ChunkMonitor.java:105)
      at org.springframework.batch.item.support.CompositeIt emStream.open(CompositeItemStream.java:98)
      at org.springframework.batch.core.step.tasklet.Taskle tStep.open(TaskletStep.java:288)
      at org.springframework.batch.core.step.AbstractStep.e xecute(AbstractStep.java:193)
      at org.springframework.batch.core.job.SimpleStepHandl er.handleStep(SimpleStepHandler.java:135)
      at org.springframework.batch.core.job.flow.JobFlowExe cutor.executeStep(JobFlowExecutor.java:61)
      at org.springframework.batch.core.job.flow.support.st ate.StepState.handle(StepState.java:60)
      at org.springframework.batch.core.job.flow.support.Si mpleFlow.resume(SimpleFlow.java:144)
      at org.springframework.batch.core.job.flow.support.Si mpleFlow.start(SimpleFlow.java:124)
      at org.springframework.batch.core.job.flow.FlowJob.do Execute(FlowJob.java:135)
      at org.springframework.batch.core.job.AbstractJob.exe cute(AbstractJob.java:281)
      at org.springframework.batch.core.launch.support.Simp leJobLauncher$1.run(SimpleJobLauncher.java:120)
      at org.springframework.core.task.SyncTaskExecutor.exe cute(SyncTaskExecutor.java:49)
      at org.springframework.batch.core.launch.support.Simp leJobLauncher.run(SimpleJobLauncher.java:114)
      at org.springframework.batch.core.launch.support.Simp leJobOperator.start(SimpleJobOperator.java:300)
      at org.springframework.transaction.support.Transactio nTemplate.execute(TransactionTemplate.java:128)
      at sun.reflect.GeneratedMethodAccessor71.invoke(Unkno wn Source)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(De legatingMethodAccessorImpl.java:43)
      at java.lang.reflect.Method.invoke(Method.java:618)
      at org.springframework.aop.support.AopUtils.invokeJoi npointUsingReflection(AopUtils.java:310)
      at org.springframework.aop.framework.ReflectiveMethod Invocation.invokeJoinpoint(ReflectiveMethodInvocat ion.java:182)
      at org.springframework.aop.framework.ReflectiveMethod Invocation.proceed(ReflectiveMethodInvocation.java :149)
      at org.springframework.transaction.interceptor.Transa ctionInterceptor.invoke(TransactionInterceptor.jav a:106)
      at org.springframework.aop.framework.ReflectiveMethod Invocation.proceed(ReflectiveMethodInvocation.java :171)
      at org.springframework.aop.interceptor.ExposeInvocati onInterceptor.invoke(ExposeInvocationInterceptor.j ava:89)
      at org.springframework.aop.framework.ReflectiveMethod Invocation.proceed(ReflectiveMethodInvocation.java :171)
      at org.springframework.aop.framework.JdkDynamicAopPro xy.invoke(JdkDynamicAopProxy.java:204)
      at $Proxy14.runInTransaction(Unknown Source)
      at EDU.oswego.cs.dl.util.concurrent.PooledExecutor$Wo rker.run(PooledExecutor.java:747)
      at java.lang.Thread.run(Thread.java:810)
      Caused by: org.springframework.batch.item.file.FlatFileParseE xception: Unable to read from resource: [URL [ftp://ProDoc/in/testmode32.txt]]
      at org.springframework.batch.item.file.FlatFileItemRe ader.readLine(FlatFileItemReader.java:226)
      at org.springframework.batch.item.file.FlatFileItemRe ader.doOpen(FlatFileItemReader.java:274)
      at org.springframework.batch.item.support.AbstractIte mCountingItemStreamItemReader.open(AbstractItemCou ntingItemStreamItemReader.java:134)
      ... 59 more
      Caused by: sun.io.MalformedInputException
      at sun.io.ByteToCharUTF8.convert(ByteToCharUTF8.java: 278)
      at sun.nio.cs.StreamDecoder$ConverterSD.convertInto(S treamDecoder.java:315)
      at sun.nio.cs.StreamDecoder$ConverterSD.implRead(Stre amDecoder.java:365)
      at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:2 51)
      at java.io.InputStreamReader.read(InputStreamReader.j ava:212)
      at java.io.BufferedReader.fill(BufferedReader.java:15 7)
      at java.io.BufferedReader.readLine(BufferedReader.jav a:320)
      at java.io.BufferedReader.readLine(BufferedReader.jav a:383)
      at org.springframework.batch.item.file.FlatFileItemRe ader.readLine(FlatFileItemReader.java:210)
      ... 61 more


      Also i am able to read the same file if i am using my local server i.e. Tomcat5.5 in eclipse but when i am trying to execute this job from Websphere (WAS6) its failing . And if i remove the characters like , from the file then in Websphere also i am able to read the file .
      Also i have tried both ways by removing and adding the property encoding as UTF-8 , but still its not working in WAS6. Everything works fine when i am using my local server Tomcat even with special characters , only case that doesnt work is special characters like , in file and i am using Wepbsphere for executing the job.

      Please let me know if its not clear .

      thanks and regards
      Sachin

      Comment


      • #4
        I can't think of anything in Spring Batch (or in the Java APIs) that would cause this. Is it a firewall mangling your file as it comes in over FTP?

        Comment


        • #5
          Hello Dave

          I am not sure whether this is a firewall problem , because if firewall is the problem then it should also give the problem when i am reading the file from FTP using my Local server Tomcat using eclipse...
          Its giving the problem only when i am trying to read this file from FTP using my Websphere (application deployed on WAS 6) and that too when its having these special characters like , in the file . So i think its something to do with these special characters(, ) in the file but i dont know how to manage this in reader bean .


          FTP File with Special characters Server Result
          .................................................. ..............................
          No Tomcat Works fine
          Yes (, ) Tomcat Works Fine
          No Websphere Works fine
          Yes (, ) Websphere fails to read .



          So the problem comes only in the last case as mentioned above and that too i think its because of some UTF-8 format . But i dont have any idea how to tackle this in Spring batch . Please help me out in case you have any idea about this kind of problem .


          Thanks and regards
          sachin

          Comment


          • #6
            The error seems to indicate that the file you get from FTP in the WAS application is not in fact UTF-8 encoded. If it is the same file, then the only thing I could think of was an external effect. You seem to have done everything you can with the application. Maybe a JVM bug? Is the JVM different in WAS?

            Comment


            • #7
              Yes, the JVM is different in WAS server . But can we do anything to handle this case ?

              Comment


              • #8
                I'm not sure what you want Batch to do about this. If you suspect the JVM, you should run a test outside WAS on a local file. I guess you can also try and verify that the bytes in the stream you get in WAS are the same as the bytes in the local app, then you can see if the encoding is changed, or if the JVM simply doesn't work with UTF-8 (seems unlikely but maybe a bug in java.io.* could cause that).

                Comment

                Working...
                X