重新编译hadoop 32bit-64bit

先安装macports

参考：http://blog.csdn.net/bamuta/article/details/13506893
下载程序代码
机器得连网，如果没联网找可以联网的机器下载，但是编译时还是要下载一些东西，所以，实在不行。最好找相同平台（可以是虚拟机）能上网的机器做下面工作，弄好了再拷回来。
svn cohttp://svn.apache.org/repos/asf/hadoop/common/tags/release-2.2.0
都下载到这了：
[hadoop@hadoop01 hadoop]$ ls
BUILDING.txt  hadoop-common-project  hadoop-maven-plugins  hadoop-tools
dev-support  hadoop-dist hadoop-minicluster  hadoop-yarn-project
hadoop-assemblies  hadoop-hdfs-project  hadoop-project  pom.xml
hadoop-client  hadoop-mapreduce-project  hadoop-project-dist
安装开发环境
1.必要的包
sudo port install libtool
sudo port install cmake
sudo port install ncurses
sudo port install openssl
sudo port install maven3  

安装java7
偶是小白，今天被人科普了
JDK是可以进行编译的，而jre只能用来运行，反正在java上要注意区分2者的区别，以后详细学习java的时候再进一步了解吧。

官网下载mac版本的安装文件
官网地址： http://www.oracle.com/technetwork/java/javase/downloads/index.html
选择jdk7的Mac OS X x64版本，jdk6没有Mac OS X版本的
下载对应的dmg文件安装
安装好在命令行输入：java -version 现实下面的内容
java version "1.7.0_17" Java(TM) SE Runtime Environment (build 1.7.0_17-b02) Java HotSpot(TM) 64-Bit Server VM (build 23.7-b01, mixed mode)
java就安装完成了，也无需配置环境变量
删除的方法也很简单，在命令行中到 /Library/Java/JavaVirtualMachines 这个目录下，删除对应的jdk就可以了

修改.bash_profile，添加
export PATH=${PATH}:/Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home
export JAVA_HOME=/Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home  

gcc也很重要
gcc -v
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/include/c++/4.2.1
Apple LLVM version 5.0 (clang-500.2.79) (based on LLVM 3.3svn)
Target: x86_64-apple-darwin13.0.0
Thread model: posix

mvn3 -v
Apache Maven 3.0.5 (r01de14724cdef164cd33c7c8c2fe155faf9602da; 2013-02-19 21:51:28+0800)
Maven home: /opt/local/share/java/maven3
Java version: 1.7.0_45, vendor: Oracle Corporation
Java home: /Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/jre
Default locale: zh_CN, platform encoding: UTF-8
OS name: "mac os x", version: "10.9", arch: "x86_64", family: "mac"

qqqq

mvn3 package -Pdist,native -DskipTests -Dtar

报错：
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 15.951s
[INFO] Finished at: Fri Dec 13 18:06:44 CST 2013
[INFO] Final Memory: 39M/525M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:2.5.1:testCompile (default-testCompile) on project hadoop-auth: Compilation failure: Compilation failure:
[ERROR] /Users/JuneMAC/hadoop/release-2.2.0/hadoop-common-project/hadoop-auth/src/test/java/org/apache/hadoop/security/authentication/client/AuthenticatorTestCase.java:[88,11] 错误: 无法访问AbstractLifeCycle
[ERROR] 找不到org.mortbay.component.AbstractLifeCycle的类文件
[ERROR] /Users/JuneMAC/hadoop/release-2.2.0/hadoop-common-project/hadoop-auth/src/test/java/org/apache/hadoop/security/authentication/client/AuthenticatorTestCase.java:[96,29] 错误: 无法访问LifeCycle
[ERROR] 找不到org.mortbay.component.LifeCycle的类文件
[ERROR] /Users/JuneMAC/hadoop/release-2.2.0/hadoop-common-project/hadoop-auth/src/test/java/org/apache/hadoop/security/authentication/client/AuthenticatorTestCase.java:[98,10] 错误: 找不到符号
[ERROR] 符号:  方法 start()
[ERROR] 位置: 类型为Server的变量 server
[ERROR] /Users/JuneMAC/hadoop/release-2.2.0/hadoop-common-project/hadoop-auth/src/test/java/org/apache/hadoop/security/authentication/client/AuthenticatorTestCase.java:[104,12] 错误: 找不到符号
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1]http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]  mvn -rf :hadoop-auth

相关讨论改错误的链接：
https://issues.apache.org/jira/browse/HADOOP-10110

解决方案：
vi hadoop-common-project/hadoop-auth/pom.xml
在

org.mortbay.jetty
jetty
test

这段之前加一段：

org.mortbay.jetty
jetty-util
test

mvn3 package -Pdist,native -DskipTests -Dtar
hadoop-auth终于成功了，然后报错：
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.hadoop:hadoop-maven-plugins:2.2.0:protoc (compile-protoc) on project hadoop-common: org.apache.maven.plugin.MojoExecutionException: 'protoc --version' did not return a version -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1]http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]  mvn -rf :hadoop-common
看到：http://blog.csdn.net/bamuta/article/details/13506893
要先装protobuf，但是macports找不到这个安装，貌似只能手动安装
在https://code.google.com/p/protobuf/downloads/list下载protobuf-2.5.0.tar.gz
在/opt/local/var/macports/software/下建立protobuf文件夹
在这个文件夹下解压缩protobuf-2.5.0.tar.gz
sudo tar -zxvfprotobuf-2.5.0.tar.gz
sudo sh configure
sudomake
sudo make check
sudo make install
protoc --version  //libprotoc 2.5.0
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib
bin的路径：/usr/local/bin

再次 mvn3 package -Pdist,native -DskipTests -Dtar
新错误：
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1:33.033s
[INFO] Finished at: Fri Dec 13 22:49:22 CST 2013
[INFO] Final Memory: 45M/350M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.6:run (make) on project hadoop-common: An Ant BuildException has occured: exec returned: 2 -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1]http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]  mvn -rf :hadoop-common
根据http://blog.csdn.net/laozitianxia/article/details/8786922
安装forrest
http://forrest.apache.org/mirrors.cgi
下载：apache-forrest-0.9-sources.tar.gzapache-forrest-0.9-dependencies.tar.gz
解压apache-forrest-0.9-sources.tar.gz
然后将apache-forrest-0.9-dependencies.tar.gz解压后，tools/下的文件整合进
/opt/local/var/macports/software/apache-forrest/apache-forrest-0.9/tools/文件夹下
修改forrest环境变量export FORREST_HOME=/opt/local/var/macports/software/apache-forrest/apache-forrest-0.9
export PATH=$PATH:$FORREST_HOME/bin
forrest -version
Apache Forrest.  Run 'forrest -projecthelp' to list options
Apache Ant version 1.7.1 compiled on September 26 2008  
再次 mvn3 package -Pdist,native -DskipTests -Dtar
新错误：
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1:56.366s
[INFO] Finished at: Sat Dec 14 00:59:51 CST 2013
[INFO] Final Memory: 58M/442M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.6:run (make) on project hadoop-common: An Ant BuildException has occured: exec returned: 2 -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1]http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]  mvn -rf :hadoop-common
根据http://blog.csdn.net/laozitianxia/article/details/8786922
安装findbugs
在http://findbugs.sourceforge.net/downloads.html下载findbugs-2.0.3.tar.gz
解压缩貌似直接能用，配置下环境变量
export FINDBUGS_HOME=/opt/local/var/macports/software/findbugs/findbugs-2.0.3
export PATH=$PATH:$FINDBUGS_HOME/bin
findbugs -version
2.0.3

安装findbugs
在http://findbugs.sourceforge.net/downloads.html下载findbugs-2.0.3.tar.gz
解压缩貌似直接能用，配置下环境变量
export FINDBUGS_HOME=/opt/local/var/macports/software/findbugs/findbugs-2.0.3
export PATH=$PATH:$FINDBUGS_HOME/bin
重启终端
findbugs -version
2.0.3

安装ANT
在http://ant.apache.org/bindownload.cgi下载apache-ant-1.9.2-bin.tar.gz
配置下环境变量
# Setting Path for Ant
export ANT_HOME=/opt/local/var/macports/software/apache-ant/apache-ant-1.9.2
export PATH=$PATH:$ANT_HOME/bin  
重启终端
ant -version
Apache Ant(TM) version 1.9.2 compiled on July 8 2013
再次sudomvn3 package -Pdist,native -DskipTests -Dtar
新错误：
/bin/sh: /Users/JuneMAC/hadoop/release-2.2.0/hadoop-common-project/hadoop-common/${env.JAVA_HOME}/bin/javah: bad substitution
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 29.800s
[INFO] Finished at: Sat Dec 14 20:14:22 CST 2013
[INFO] Final Memory: 36M/96M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.codehaus.mojo:native-maven-plugin:1.0-alpha-7:javah (default) on project hadoop-common: Error running javah command: Error executing command line. Exit code:1 -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1]http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]  mvn -rf :hadoop-common
修改
hadoop-common-project/hadoop-common/pom.xml 文件中，将env.JAVA_HOME改为java.home
再次sudomvn3 package -Pdist,native -DskipTests -Dtar
新错误：

[exec] /Users/JuneMAC/hadoop/release-2.2.0/hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/security/JniBasedUnixGroupsNetgroupMapping.c:77:26: error: invalid operands to binary expression_r('void' and 'int')
[exec]  if(setnetgrent(cgroup) == 1) {
[exec]  ~~~~~~~~~~~~~~~~~~~ ^  ~
[exec] 1 error generated.
[exec] make[2]: *** [CMakeFiles/hadoop.dir/main/native/src/org/apache/hadoop/security/JniBasedUnixGroupsNetgroupMapping.c.o] Error 1
[exec] make[1]: *** [CMakeFiles/hadoop.dir/all] Error 2
[exec] make: *** [all] Error 2
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Apache Hadoop Main ................................ SUCCESS [1.455s]
[INFO] Apache Hadoop Project POM ......................... SUCCESS [1.183s]
[INFO] Apache Hadoop Annotations ......................... SUCCESS [2.488s]
[INFO] Apache Hadoop Assemblies .......................... SUCCESS [0.400s]
[INFO] Apache Hadoop Project Dist POM .................... SUCCESS [2.396s]
[INFO] Apache Hadoop Maven Plugins ....................... SUCCESS [3.434s]
[INFO] Apache Hadoop Auth ................................ SUCCESS [2.611s]
[INFO] Apache Hadoop Auth Examples ....................... SUCCESS [2.349s]
[INFO] Apache Hadoop Common .............................. FAILURE [10.596s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 29.226s
[INFO] Finished at: Sat Dec 14 21:07:40 CST 2013
[INFO] Final Memory: 36M/94M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.6:run (make) on project hadoop-common: An Ant BuildException has occured: exec returned: 2 -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1]http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]  mvn -rf :hadoop-common
在hadoop群里求教之后采用如下方案：修改mvn3的配置文件：/opt/local/share/java/maven3/settings.xml
在…里添加国内源：



nexus-osc
*
Nexusosc
http://maven.oschina.net/content/groups/public/



在...标签中增加以下内容：

jdk-1.7

1.7



nexus
local private nexus
http://maven.oschina.net/content/groups/public/

true


false





nexus
local private nexus
http://maven.oschina.net/content/groups/public/

true


false






注意修改jdk version number

将刚才的maven 配置文件拷贝到当前用户的home目录下:
settings.xml  copy 到  your_hadoop_usr_home/.m2/
cp settings.xml ~/.m2

最重要的一点，build your code是使用这个command line(Only for Mac OS):
mvn3 clean install -P-cbuild

编译之前， 你在hadoop-2.2.0-src目录（/Users/JuneMAC/hadoop/release-2.2.0）下执行
mvn3 clean install –DskipTests

上面的成功后，执行下面这个，生成安装包
mvn3 clean install  package -Pdist -P-cbuild  -DskipTests  -Dtar

执行完成后，可以在/Users/JuneMAC/hadoop/release-2.2.0/hadoop-dist/target/
下找到
hadoop-2.2.0.tar.gz
将上面这个编译好的源码包解压到：
/Users/JuneMAC/hadoop/
然后进行相关配置
解压之后的源码包和官网下载下来的源码包相对比，没有lib目录
相关解释：
“Here we use the additional options to stop compiling the native code. Here we use the additional options to stop compiling the native code.
this is the key reason why we need use -P-cbuild option”
上面这个是原因，好像不是很重要，这点先pass了，以后遇到问题再来补充说明吧

附:(官方说明)

Index: hadoop-common-project/hadoop-auth/pom.xml
===================================================================
--- hadoop-common-project/hadoop-auth/pom.xml	(revision 1543124)
+++ hadoop-common-project/hadoop-auth/pom.xml	(working copy)
@@ -54,6 +54,11 @@
     </dependency>
     <dependency>
       <groupId>org.mortbay.jetty</groupId>
+      <artifactId>jetty-util</artifactId>
+      <scope>test</scope>
+    </dependency>
+    <dependency>
+      <groupId>org.mortbay.jetty</groupId>
       <artifactId>jetty</artifactId>
       <scope>test</scope>
     </dependency>

详细请查看:https://issues.apache.org/jira/browse/HADOOP-10110