`
superlxw1234
  • 浏览: 543394 次
  • 性别: Icon_minigender_1
  • 来自: 西安
博客专栏
Bd1c0a0c-379a-31a8-a3b1-e6401e2f1523
Hive入门
浏览量:43384
社区版块
存档分类
最新评论

Cygwin中运行hadoop mapreduce任务必须设置中间暂存目录

 
阅读更多

cygwin上运行hadoop mr任务,一直报错:

 

2012-04-20 00:47:28,937 WARN org.apache.hadoop.mapred.TaskTracker: Error running child
java.io.FileNotFoundException: File C:/home/Administrator/hadoop-0.20.2/hadooptmp/mapred/local/taskTracker/jobcache/job_201204200043_0002/attempt_201204200043_0002_m_000001_0/work/tmp does not exist.
	at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:361)
	at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:245)
	at org.apache.hadoop.mapred.TaskRunner.setupWorkDir(TaskRunner.java:519)
	at org.apache.hadoop.mapred.Child.main(Child.java:155)
2012-04-20 00:47:28,937 INFO org.apache.hadoop.mapred.TaskRunner: Runnning cleanup for the task
2012-04-20 00:47:28,937 INFO org.apache.hadoop.mapred.TaskTracker: Error cleaning upjava.lang.NullPointerException

 

 

最终解决:

在mapred-site.xml中添加:

<property>
  <name>mapred.child.tmp</name>
  <value>/home/Administrator/hadoop-0.20.2/child/temp</value>
  <description> To set the value of tmp directory for map and reduce tasks.
  If the value is an absolute path, it is directly assigned. Otherwise, it is
  prepended with task's working directory. The java tasks are executed with
  option -Djava.io.tmpdir='the absolute path of the tmp dir'. Pipes and
  streaming are set with environment variable,
   TMPDIR='the absolute path of the tmp dir'
  </description>
</property>
 
分享到:
评论

相关推荐

Global site tag (gtag.js) - Google Analytics