Hadoop学习-错误记录：TokenizerMapper not found

77 阅读 0 评论 51 点赞

我是靠谱客的博主粗心楼房，最近开发中收集的这篇文章主要介绍Hadoop学习-错误记录：TokenizerMapper not found，觉得挺不错的，现在分享给大家，希望可以做个参考。

概述

在运行WordCount程序时，报如题错误信息，在网上苦寻解决方案无果后，终于自己排查出了错误。发表在此给需要的人一个参考。
报错信息：

17/09/14 06:17:11 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
17/09/14 06:17:12 INFO client.RMProxy: Connecting to ResourceManager at master/192.168.137.129:8032
17/09/14 06:17:13 WARN mapreduce.JobResourceUploader: Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this.
17/09/14 06:17:13 WARN mapreduce.JobResourceUploader: No job jar file set.  User classes may not be found. See Job or Job#setJar(String).
17/09/14 06:17:13 INFO input.FileInputFormat: Total input paths to process : 9
17/09/14 06:17:13 INFO mapreduce.JobSubmitter: number of splits:9
17/09/14 06:17:13 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1505394411892_0002
17/09/14 06:17:13 INFO mapred.YARNRunner: Job jar is not present. Not adding any jar to the list of resources.
17/09/14 06:17:13 INFO impl.YarnClientImpl: Submitted application application_1505394411892_0002
17/09/14 06:17:13 INFO mapreduce.Job: The url to track the job: http://master:8088/proxy/application_1505394411892_0002/
17/09/14 06:17:13 INFO mapreduce.Job: Running job: job_1505394411892_0002
17/09/14 06:17:22 INFO mapreduce.Job: Job job_1505394411892_0002 running in uber mode : false
17/09/14 06:17:22 INFO mapreduce.Job:  map 0% reduce 0%
17/09/14 06:17:26 INFO mapreduce.Job: Task Id : attempt_1505394411892_0002_m_000008_0, Status : FAILED
Error: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class wordCount.WordCount$TokenizerMapper not found
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2195)
    at org.apache.hadoop.mapreduce.task.JobContextImpl.getMapperClass(JobContextImpl.java:186)
    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:745)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
    at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
    at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: java.lang.ClassNotFoundException: Class wordCount.WordCount$TokenizerMapper not found
    at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2101)
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2193)
    ... 8 more

17/09/14 06:17:30 INFO mapreduce.Job: Task Id : attempt_1505394411892_0002_m_000008_1, Status : FAILED
Error: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class wordCount.WordCount$TokenizerMapper not found
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2195)
    at org.apache.hadoop.mapreduce.task.JobContextImpl.getMapperClass(JobContextImpl.java:186)
    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:745)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
    at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
    at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: java.lang.ClassNotFoundException: Class wordCount.WordCount$TokenizerMapper not found
    at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2101)
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2193)
    ... 8 more

报错原因及修正方法：在配置eclipse项目时，按照网上的教程提示，将所有的修改过的/etc/hadoop中的文件放到了eclipse工程src目录下面。
由于我撘的环境是全分布式集群，修改了core-site.xml,hdfs-site.xml,mapred-site.xml,slaves,yarn-site.xml这五个文件，所以我将这五个文件以及log4j.properties全部放到了src目录下，但是实际上只需要core-site.xml和hdfs-site.xml两个文件以及log4j.properties。
在删除多余的三个文件后，再次运行程序不再报错。

ps：具体为什么这三个文件会导致报TokenizerMapper not found错误目前尚不清楚，如果有大牛知道还请赐教！
pps：多种原因可能导致程序报这个错误，细心排查最重要。