Announcement Announcement Module
Collapse
No announcement yet.
MR class not found exception in WordCount example use spring hadoop Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • MR class not found exception in WordCount example use spring hadoop

    HI:
    when I execute the WordCount example use hadoop spring,it always display "Class Not Found Exception:TokenizerMapper",but I have already put the hadoop-examples-1.0.3.jar under the libraries of my project,and my context.xml as follow:


    <?xml version="1.0" encoding="UTF-8"?>
    <beans xmlns="http://www.springframework.org/schema/beans"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xmlns:context="http://www.springframework.org/schema/context"
    xmlns:hdp="http://www.springframework.org/schema/hadoop"
    xmlns="http://www.springframework.org/schema/p"
    xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schem...ring-beans.xsd
    http://www.springframework.org/schema/context http://www.springframework.org/schem...ng-context.xsd
    http://www.springframework.org/schema/hadoop http://www.springframework.org/schema/hadoop/spring-hadoop.xsd">
    <contextroperty-placeholder location="hadoop.properties"/>
    <hdp:configuration>
    fs.default.name=${hd.fs}
    </hdp:configuration>
    <hdp:job id="wordcount-job" validate-paths="false"
    input-path="${wordcount.input.path}" output-path="${wordcount.output.path}"
    mapper="org.apache.hadoop.examples.WordCount.Token izerMapper"
    reducer="org.apache.hadoop.examples.WordCount.IntS umReducer"
    />
    <hdp:script id="clean-script" language="javascript" run-at-startup="true">
    inputPath = "${wordcount.input.path}"
    outputPath = "${wordcount.output.path}"
    if (fsh.test(inputPath)) { fsh.rmr(inputPath) }
    if (fsh.test(outputPath)) { fsh.rmr(outputPath) }
    inStream = cl.getResourceAsStream("nietzsche-chapter-1.txt")
    org.apache.hadoop.io.IOUtils.copyBytes(inStream, fs.create(inputPath), cfg)
    </hdp:script>
    <bean id="runner" class="org.springframework.data.hadoop.mapreduce.J obRunner" depends-on="clean-script" p:jobs-ref="wordcount-job"/>
    </beans>


    who can tell me what's the matter?

  • #2
    Replied to your other post on this as well. I think the solution is to add the hadoop-examples dependency:

    You may need to add the hadoop-examples dependency which has those example classes.

    <dependency>
    <groupId>org.apache.hadoop</groupId>
    <artifactId>hadoop-examples</artifactId>
    </dependency>

    Comment

    Working...
    X