Announcement Announcement Module
Collapse
No announcement yet.
Hadoop tool-tasklet fails with space in jar path on Windows Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • Hadoop tool-tasklet fails with space in jar path on Windows

    I am wondering if someone knows a workaround for this issue I've been encountering...

    I am setting up a Spring Batch workflow, using "hdp:tool-tasklet" with my hadoop job jar file. When the workflow triggers the job to be executed, it fails saying it can't find the job Jar. It looks like it has something to do with the way the path is converted to a URI and gets character escaped. I confirmed that if I copy the Jar to a path with no spaces and add that new path to the classpath, it works as expected.

    For example, if I have my jar in the path "c:\Program Files\myfolder\job.jar" I will get the following exception:

    2013-07-26 09:38:07 ERROR AbstractStep:222 - Encountered an error executing the step
    java.io.FileNotFoundException: File file:/C:/Program%20Files/myfolder/job.jar does not exist.
    at org.apache.hadoop.fs.RawLocalFileSystem.getFileSta tus(RawLocalFileSystem.java:394)
    at org.apache.hadoop.fs.FilterFileSystem.getFileStatu s(FilterFileSystem.java:251)
    at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:2 07)
    at org.apache.hadoop.fs.FileSystem.copyFromLocalFile( FileSystem.java:1200)
    at org.apache.hadoop.fs.FileSystem.copyFromLocalFile( FileSystem.java:1166)
    at org.apache.hadoop.fs.FileSystem.copyFromLocalFile( FileSystem.java:1138)
    at org.apache.hadoop.mapred.JobClient.copyAndConfigur eFiles(JobClient.java:798)

    My tasklet is configured like:

    <hdp:tool-tasklet id="myTasklet" tool-class="com.test.myclass" jar="job.jar" />
Working...
X