Announcement Announcement Module
Collapse
No announcement yet.
When I run the WordCount example,it always display Class Not Found Exception Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • When I run the WordCount example,it always display Class Not Found Exception

    Hello
    When I run the WordCount example use spring hadoop,it always display Class Not Found Exception as follow:

    java.lang.RuntimeException: java.lang.ClassNotFoundException: org.apache.hadoop.examples.WordCount$TokenizerMapp er
    at org.apache.hadoop.conf.Configuration.getClass(Conf iguration.java:867)
    at org.apache.hadoop.mapreduce.JobContext.getMapperCl ass(JobContext.java:199)
    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapT ask.java:719)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java: 370)
    at org.apache.hadoop.mapred.Child$4.run(Child.java:25 5)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:396)
    at org.apache.hadoop.security.UserGroupInformation.do As(UserGroupInformation.java:1121)
    at org.apache.hadoop.mapred.Child.main(Child.java:249 )
    Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.examples.WordCount$TokenizerMapp er
    at java.net.URLClassLoader$1.run(URLClassLoader.java: 202)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.j ava:190)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:3 06)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launche r.java:301)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:2 47)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:247)
    at org.apache.hadoop.conf.Configuration.getClassByNam e(Configuration.java:820)
    at org.apache.hadoop.conf.Configuration.getClass(Conf iguration.java:865)
    ... 8 more


    And my context.xml as follow:


    <?xml version="1.0" encoding="UTF-8"?>
    <beans xmlns="http://www.springframework.org/schema/beans"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xmlns:context="http://www.springframework.org/schema/context"
    xmlns:hdp="http://www.springframework.org/schema/hadoop"
    xmlns="http://www.springframework.org/schema/p"
    xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schem...ring-beans.xsd
    http://www.springframework.org/schema/context http://www.springframework.org/schem...ng-context.xsd
    http://www.springframework.org/schema/hadoop http://www.springframework.org/schema/hadoop/spring-hadoop.xsd">
    <contextroperty-placeholder location="hadoop.properties"/>
    <hdp:configuration>
    fs.default.name=${hd.fs}
    </hdp:configuration>

    <hdp:job id="wordcount-job" validate-paths="false"
    input-path="${wordcount.input.path}" output-path="${wordcount.output.path}"
    mapper="org.apache.hadoop.examples.WordCount.Token izerMapper"
    reducer="org.apache.hadoop.examples.WordCount.IntS umReducer"
    />


    <hdp:script id="clean-script" language="javascript" run-at-startup="true">
    inputPath = "${wordcount.input.path}"
    outputPath = "${wordcount.output.path}"
    if (fsh.test(inputPath)) { fsh.rmr(inputPath) }
    if (fsh.test(outputPath)) { fsh.rmr(outputPath) }
    inStream = cl.getResourceAsStream("nietzsche-chapter-1.txt")
    org.apache.hadoop.io.IOUtils.copyBytes(inStream, fs.create(inputPath), cfg)
    </hdp:script>
    <bean id="runner" class="org.springframework.data.hadoop.mapreduce.J obRunner" depends-on="clean-script" p:jobs-ref="wordcount-job"/>
    </beans>

    please help me to check the problem,thank you verry much!

  • #2
    You may need to add the hadoop-examples dependency which has those example classes.

    <dependency>
    <groupId>org.apache.hadoop</groupId>
    <artifactId>hadoop-examples</artifactId>
    </dependency>

    Comment

    Working...
    X