您好,登录后才能下订单哦!
IDEA 开发 WordCount(Spark)报错
WordCount { } WordCount{ (args: Array[]) { hadoopAddr = conf = SparkConf().setAppName() sc = SparkContext(conf) mapRdd = sc.textFile(hadoopAddr) result = mapRdd.flatMap(_.split()).map((_)).reduceByKey(_ + _).collect() } }
Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
15/12/08 22:07:16 INFO SparkContext: Running Spark version 1.4.0
15/12/08 22:07:17 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
15/12/08 22:07:18 ERROR Shell: Failed to locate the winutils binary in the hadoop binary path
java.io.IOException: Could not locate executable null\bin\winutils.exe in the Hadoop binaries.
at org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.java:355)
at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:370)
at org.apache.hadoop.util.Shell.<clinit>(Shell.java:363)
at org.apache.hadoop.util.StringUtils.<clinit>(StringUtils.java:79)
at org.apache.hadoop.security.Groups.parseStaticMapping(Groups.java:104)
at org.apache.hadoop.security.Groups.<init>(Groups.java:86)
at org.apache.hadoop.security.Groups.<init>(Groups.java:66)
at org.apache.hadoop.security.Groups.getUserToGroupsMappingService(Groups.java:280)
免责声明:本站发布的内容(图片、视频和文字)以原创、转载和分享为主,文章观点不代表本网站立场,如果涉及侵权请联系站长邮箱:is@yisu.com进行举报,并提供相关证据,一经查实,将立刻删除涉嫌侵权内容。