Hadoop2.4.0 Eclipse插件制作

浪打天門 2014-07-21

展開全文

Hadoop2.4.0 Eclipse插件制作

時(shí)間2014-04-16 19:29:48 CSDN博客原文 http://blog.csdn.net/yueritian/article/details/23868175

跌跌撞撞兩天裝Hadoop2.4.0的環(huán)境,，今天終于可以嘗試寫點(diǎn)程序了，想了一想,，不得其法,，但有一點(diǎn)是肯定的,，那就是需要用eclipse，查了一上午的資料,，怎么講的都有,，最后鎖定了這個(gè)帖子， hadoop2x-eclipse-plugin ,。想用Github下載,，但是最后下載失敗，頁(yè)面右側(cè)有個(gè)Download ZIP,，用這個(gè)下載成功,。下邊來(lái)說說我的整個(gè)制作過程吧：

想導(dǎo)入到eclipse中制作，但是導(dǎo)入進(jìn)去之后覺得麻煩就算了,，直接在win7 64位,、JDK1.6、ANT1.8的環(huán)境下以命令行運(yùn)行
依葫蘆畫瓢,，設(shè)置eclipse的目錄和hadoop的目錄,，在沒有修改任何文件的情況下直接運(yùn)行

D:\SDK\hadoop2x-eclipse-plugin-master\src\contrib\eclipse-plugin>ant jar -Dversion=2.4.0 -Declipse.home=D:\WorkTools\MyEclipse10\Common -Dhadoop.home=D:\SDK\hadoop-2.4.0

在ivy-resolve-common處卡了大概138分鐘，最后build失敗,，查看原因有兩個(gè),，一個(gè)是找不到4個(gè)依賴包,，一個(gè)是找不到1個(gè)核心包
依賴包分別是這些，可能是換路徑了吧：

https://oss.sonatype.org/content/groups/public/com/google/google/1/google-1.jar
https://oss.sonatype.org/content/groups/public/net/java/jvnet-parent/1/jvnet-parent-1.jar
https://oss.sonatype.org/content/groups/public/com/sun/jersey/contribs/jersey-contribs/1.9/jersey-contribs-1.9.jar
https://oss.sonatype.org/content/groups/public/org/eclipse/jetty/jetty-parent/14/jetty-parent-14.jar

核心包是commons-lang-2.5.jar
想了一下,，覺得那四個(gè)依賴包似乎沒什么用，便想跳過那個(gè)步驟,，想直接進(jìn)行編譯插件的步驟,，便修改了“hadoop2x-eclipse-plugin-master\src\contrib\eclipse-plugin\build.xml”文件

<target name="compile" depends="init, ivy-retrieve-common" unless="skip.contrib">     去掉depends

核心包去hadoop2.4目錄找，發(fā)現(xiàn)hadoop2.4用的commons-lang-2.6.jar,，便在build.xml中找到相應(yīng)位置改為2.6,，這個(gè)需要說的是作者可能偷懶，在拷貝文件中使用的是變量,，在配置信息中直接使用了2.5,，不統(tǒng)一，變量配置信息在“hadoop2x-eclipse-plugin-master\ivy\libraries.properties”文件中,，搜索“commons-lang.version”將對(duì)應(yīng)的版本改為2.6,，好了，在命令行下運(yùn)行ant,，成功在“hadoop2x-eclipse-plugin-master\build\contrib\eclipse-plugin”下生成插件
興奮的拿著插件跟myeclipse10里試,，失敗（插件壓根不出現(xiàn)）,，跟eclipse3.7里試,，失敗（插件能出現(xiàn),，但是無(wú)法正常使用,，新增不了location）
拿著現(xiàn)成的2.2的插件均可以正常使用
后來(lái)經(jīng)過排查原因myeclipse10里是因?yàn)椴寮诺奈恢貌粚?duì)，不能直接放到plugin里,；eclipse3.7里失敗是因?yàn)椴寮€缺少包
如何排查的呢,？看eclipse的啟動(dòng)日志，這個(gè)日志在workplace中的.metadata文件下的.log文件,，在eclipse3.7里報(bào)的錯(cuò)誤是無(wú)法初始化hadoop,，很郁悶，沒太大幫助,，于是就上eclilpse的官網(wǎng)下載了最新版的eclipse4.3.2，將ant語(yǔ)句改為

D:\SDK\hadoop2x-eclipse-plugin-master\src\contrib\eclipse-plugin>ant jar -Dversion=2.4.0 -Declipse.home=D:\WorkTools\eclipse4.3.2 -Dhadoop.home=D:\SDK\hadoop-2.4.0

對(duì),，改為以eclipse4.3.2為eclipse目錄(這個(gè)配置主要就是為生成eclipse插件提供jar包支持),，生成插件后放到eclipse4.3.2的plugins里，啟動(dòng)eclipse并觀察日志,，拋出異常

java.lang.NoClassDefFoundError: org/apache/commons/collections/map/UnmodifiableMap

yes~ 折騰了又兩個(gè)多小時(shí),，終于讓我抓住了,，找到這個(gè)包，在build.xml中添加上,，別忘了在配置信息中也要添加,，就是文件最下邊的attribute，如果想以后方便,，版本號(hào)也可以通過變量形式在libraries.properties中維護(hù),，搞定后，重新運(yùn)行ant
終于行了,，在myeclipse10下也可正常使用了（這么建立文件夾MyEclipse 10\dropins\hadoop\eclipse\plugins,，把插件扔這個(gè)文件夾下邊就行了）

------------------------------------------------最終的build.xml文件，修改的地方有注釋---------------------------------------------------------------

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<project default="jar" name="eclipse-plugin">
  <import file="../build-contrib.xml"/>
  <path id="eclipse-sdk-jars">
    <fileset dir="${eclipse.home}/plugins/">
      <include name="org.eclipse.ui*.jar"/>
      <include name="org.eclipse.jdt*.jar"/>
      <include name="org.eclipse.core*.jar"/>
      <include name="org.eclipse.equinox*.jar"/>
      <include name="org.eclipse.debug*.jar"/>
      <include name="org.eclipse.osgi*.jar"/>
      <include name="org.eclipse.swt*.jar"/>
      <include name="org.eclipse.jface*.jar"/>
      <include name="org.eclipse.team.cvs.ssh2*.jar"/>
      <include name="com.jcraft.jsch*.jar"/>
    </fileset> 
  </path>
  <path id="hadoop-sdk-jars">
    <fileset dir="${hadoop.home}/share/hadoop/mapreduce">
      <include name="hadoop*.jar"/>
    </fileset> 
    <fileset dir="${hadoop.home}/share/hadoop/hdfs">
      <include name="hadoop*.jar"/>
    </fileset> 
    <fileset dir="${hadoop.home}/share/hadoop/common">
      <include name="hadoop*.jar"/>
    </fileset> 
  </path>
  <path id="classpath">
    <pathelement location="${build.classes}"/>
    <path refid="eclipse-sdk-jars"/>
    <path refid="hadoop-sdk-jars"/>
  </path>
  <target name="check-contrib" unless="eclipse.home">
    <property name="skip.contrib" value="yes"/>
    <echo message="eclipse.home unset: skipping eclipse plugin"/>
  </target>
  <!-- 這里我最后刪除了depends="init, ivy-retrieve-common" -->
 <target name="compile" depends="init, ivy-retrieve-common" unless="skip.contrib">
    <echo message="contrib: ${name}"/>
    <javac
     encoding="${build.encoding}"
     srcdir="${src.dir}"
     includes="**/*.java"
     destdir="${build.classes}"
     debug="${javac.debug}"
     deprecation="${javac.deprecation}">
     <classpath refid="classpath"/>
    </javac>
  </target>
  <target name="jar" depends="compile" unless="skip.contrib">
    <mkdir dir="${build.dir}/lib"/>
    <copy  todir="${build.dir}/lib/" verbose="true">
          <fileset dir="${hadoop.home}/share/hadoop/mapreduce">
           <include name="hadoop*.jar"/>
          </fileset>
    </copy>
    <copy  todir="${build.dir}/lib/" verbose="true">
          <fileset dir="${hadoop.home}/share/hadoop/common">
           <include name="hadoop*.jar"/>
          </fileset>
    </copy>
    <copy  todir="${build.dir}/lib/" verbose="true">
          <fileset dir="${hadoop.home}/share/hadoop/hdfs">
           <include name="hadoop*.jar"/>
          </fileset>
    </copy>
    <copy  todir="${build.dir}/lib/" verbose="true">
          <fileset dir="${hadoop.home}/share/hadoop/yarn">
           <include name="hadoop*.jar"/>
          </fileset>
    </copy>

    <copy  todir="${build.dir}/classes" verbose="true">
          <fileset dir="${root}/src/java">
           <include name="*.xml"/>
          </fileset>
    </copy>
    <copy file="${hadoop.home}/share/hadoop/common/lib/protobuf-java-${protobuf.version}.jar"  todir="${build.dir}/lib" verbose="true"/>
    <copy file="${hadoop.home}/share/hadoop/common/lib/log4j-${log4j.version}.jar"  todir="${build.dir}/lib" verbose="true"/>
    <copy file="${hadoop.home}/share/hadoop/common/lib/commons-cli-${commons-cli.version}.jar"  todir="${build.dir}/lib" verbose="true"/>
    <copy file="${hadoop.home}/share/hadoop/common/lib/commons-configuration-${commons-configuration.version}.jar"  todir="${build.dir}/lib" verbose="true"/>
    <copy file="${hadoop.home}/share/hadoop/common/lib/commons-lang-${commons-lang.version}.jar"  todir="${build.dir}/lib" verbose="true"/>
    <!-- 下邊這行是新增的,，原來(lái)沒有 -->
    <copy file="${hadoop.home}/share/hadoop/common/lib/commons-collections-${commons-collections.version}.jar"  todir="${build.dir}/lib" verbose="true"/>
    <copy file="${hadoop.home}/share/hadoop/common/lib/jackson-core-asl-${jackson.version}.jar"  todir="${build.dir}/lib" verbose="true"/>
    <copy file="${hadoop.home}/share/hadoop/common/lib/jackson-mapper-asl-${jackson.version}.jar"  todir="${build.dir}/lib" verbose="true"/>
    <copy file="${hadoop.home}/share/hadoop/common/lib/slf4j-log4j12-${slf4j-log4j12.version}.jar"  todir="${build.dir}/lib" verbose="true"/>
    <copy file="${hadoop.home}/share/hadoop/common/lib/slf4j-api-${slf4j-api.version}.jar"  todir="${build.dir}/lib" verbose="true"/>
    <copy file="${hadoop.home}/share/hadoop/common/lib/guava-${guava.version}.jar"  todir="${build.dir}/lib" verbose="true"/>
    <copy file="${hadoop.home}/share/hadoop/common/lib/hadoop-auth-${hadoop.version}.jar"  todir="${build.dir}/lib" verbose="true"/>
    <copy file="${hadoop.home}/share/hadoop/common/lib/commons-cli-${commons-cli.version}.jar"  todir="${build.dir}/lib" verbose="true"/>
    <copy file="${hadoop.home}/share/hadoop/common/lib/netty-${netty.version}.jar"  todir="${build.dir}/lib" verbose="true"/>
    <jar
      jarfile="${build.dir}/hadoop-${name}-${version}.jar"
      manifest="${root}/META-INF/MANIFEST.MF">
      <manifest>
   <attribute name="Bundle-ClassPath" 
    value="classes/, 
 lib/hadoop-mapreduce-client-core-${hadoop.version}.jar,
 lib/hadoop-mapreduce-client-common-${hadoop.version}.jar,
 lib/hadoop-mapreduce-client-jobclient-${hadoop.version}.jar,
 lib/hadoop-auth-${hadoop.version}.jar,
 lib/hadoop-common-${hadoop.version}.jar,
 lib/hadoop-hdfs-${hadoop.version}.jar,
 lib/protobuf-java-${protobuf.version}.jar,
 lib/log4j-${log4j.version}.jar,
 lib/commons-cli-1.2.jar,
 lib/commons-configuration-1.6.jar,
 lib/commons-httpclient-3.1.jar,
 <!-- 下邊這行原來(lái)寫的是commons-lang-2.5 -->
 lib/commons-lang-${commons-lang.version}.jar,
 <!-- 下邊這行是新增的,，原來(lái)沒有 -->
 lib/commons-collections-${commons-collections.version}.jar,
 lib/jackson-core-asl-1.8.8.jar,
 lib/jackson-mapper-asl-1.8.8.jar,
 lib/slf4j-log4j12-1.7.5.jar,
 lib/slf4j-api-1.7.5.jar,
 lib/guava-${guava.version}.jar,
 lib/netty-${netty.version}.jar"/>
   </manifest>
      <fileset dir="${build.dir}" includes="classes/ lib/"/>
      <fileset dir="${root}" includes="resources/ plugin.xml"/>
    </jar>
  </target>
</project>

本站是提供個(gè)人知識(shí)管理的網(wǎng)絡(luò)存儲(chǔ)空間，所有內(nèi)容均由用戶發(fā)布,，不代表本站觀點(diǎn),。請(qǐng)注意甄別內(nèi)容中的聯(lián)系方式、誘導(dǎo)購(gòu)買等信息,，謹(jǐn)防詐騙,。如發(fā)現(xiàn)有害或侵權(quán)內(nèi)容，請(qǐng)點(diǎn)擊一鍵舉報(bào),。

轉(zhuǎn)藏 分享

QQ空間 QQ好友新浪微博微信

獻(xiàn)花（0） +1

來(lái)自：浪打天門 > 《hadoop》

舉報(bào)/認(rèn)領(lǐng)