hadoop-1.0.3

jdk1.6.0_32

eclipse hadoop開發(fā)插件 hadoop-eclipse-plugin-1.0.3.jar

hadoop-1.0.3 使用單機(jī)模式安裝,， 端口 hdfs 9000 mapred 9001

B)環(huán)境

1） hadoop-1.0.3 的安裝步驟 可以參照http://hadoop./docs/r1.0.3/single_node_setup.html

conf/core-site.xml

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

<!-- Put site-specific property overrides in this file. -->
<configuration>
<property>  
        <name>fs.default.name</name>  
        <value>hdfs://xxx.xxx.xxx.xxx:9000</value>  
</property>
<property>
        <name>hadoop.tmp.dir</name>
        <value>/opt/hadoop-1.0.3/tmp</value> 
</property>
</configuration>

conf/mapred-site.xml:

<configuration>
     <property>
         <name>mapred.job.tracker</name>
         <value>xxx.xxx.xxx.xxx:9001</value>
     </property>
</configuration>

conf/hdfs-site.xml

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

<!-- Put site-specific property overrides in this file. -->

<configuration>
        <property>  
                <name>dfs.replication</name>  
                <value>1</value>  
        </property>  
        <property> 
                <name>dfs.permissions</name> 
                <value>false</value> 
        </property>
</configuration>

2）開發(fā)

1,、用的是WINDOWS 下的 Eclipse，將插件復(fù)制到plugin目錄里就可以了

2,、將hadoop-1.0.3 解壓到程序員的開發(fā)機(jī)器上

3,、設(shè)置Hadoop Map/Reduce到你解壓的目錄

4、配置Map/Reduce Location

5,、

新建Map/Reduce工程

examples\org\apache\hadoop\examples中的WordCount.java導(dǎo)入到工程里

問題出來了 直接運(yùn)行的話 會(huì)提示權(quán)限問題,。

12/09/24 13:52:27 ERROR security.UserGroupInformation: PriviledgedActionException as:zzj cause:java.io.IOException: Failed to set permissions of path: \tmp\hadoop-zzj\mapred\staging\zzj-743566479\.staging to 0700
Exception in thread "main" java.io.IOException: Failed to set permissions of path: \tmp\hadoop-zzj\mapred\staging\zzj-743566479\.staging to 0700

處理這個(gè)問題可以直接使用。

conf.set("mapred.job.tracker", "218.249.75.144:9001");

用RunAS 選擇 Run on Hadoop運(yùn)行這個(gè)工程,。eclipse 會(huì)自動(dòng)將Jar包傳輸?shù)紿adoop上并運(yùn)行