概述
1.下载oozie5.0
1.1 安装基础环境
oozie5.0.0的编译安装环境需求如下：
- System Requirements:
- Unix box (tested on Mac OS X and Linux)
- Java JDK 1.8+
- Maven 3.0.1+
- Hadoop 2.6.0+
- Pig 0.10.1+
JDK commands (java, javac) must be in the command path.
The Maven command (mvn) must be in the command path.
1.2 下载oozie
官网下载, 选择一个国内镜像下载文件oozie-5.0.0.tar.gz 即可
2.编译oozie
2.1 解压文件
将oozie的安装包解压到/usr/local目录下： /usr/local/oozie-5.0.0
目录结构如下:
[root@cm-server-1 oozie-5.0.0]# ll
total 352
drwxr-xr-x 2 502 wheel 4096 Jul 25 16:23 bin
drwxr-xr-x 2 502 wheel 4096 Jul 25 16:23 builds
drwxr-xr-x 3 502 wheel 4096 Jul 25 16:23 client
drwxr-xr-x 3 502 wheel 4096 Jul 25 16:23 core
drwxr-xr-x 3 502 wheel 4096 Jul 25 16:23 distro
drwxr-xr-x 3 502 wheel 4096 Jul 25 16:23 docs
drwxr-xr-x 3 502 wheel 4096 Jul 25 16:23 examples
-rw-r--r-- 1 502 wheel 1330 Mar 27 20:22 findbugs-filter.xml
-rw-r--r-- 1 502 wheel 37664 Mar 27 20:22 LICENSE.txt
drwxr-xr-x 3 502 wheel 4096 Jul 25 16:23 minitest
-rw-r--r-- 1 502 wheel 450 Mar 27 20:22 NOTICE.txt
-rw-r--r-- 1 502 wheel 105126 Mar 27 20:22 pom.xml
-rw-r--r-- 1 502 wheel 3084 Mar 27 20:22 README.txt
-rw-r--r-- 1 502 wheel 137736 Mar 27 20:22 release-log.txt
drwxr-xr-x 3 502 wheel 4096 Jul 25 16:23 server
drwxr-xr-x 12 502 wheel 4096 Jul 25 16:23 sharelib
-rw-r--r-- 1 502 wheel 2653 Mar 27 20:22 source-headers.txt
drwxr-xr-x 3 502 wheel 4096 Mar 27 20:22 src
drwxr-xr-x 3 502 wheel 4096 Jul 25 16:23 tools
drwxr-xr-x 3 502 wheel 4096 Jul 25 16:23 webapp
drwxr-xr-x 3 502 wheel 4096 Jul 25 16:23 zookeeper-security-tests
[root@cm-server-1 oozie-5.0.0]#
2.2 编译代码
bin/mkdistro.sh -Dhadoop.version=2.6.0 -Dhbase.version=1.2.0 -Dspark.version=2.1.0 -DskipTests
（本来要添加hive的版本为1.1.0的，可以版本太低了吧编译不通过于是给去掉了，默认hive应该是1.2.0的了）
2.3 获取打包版本
编译完成之后在获取打包文件 oozie-5.0.0/distro/target/oozie-5.0.0-distro.tar.gz
3. 安装oozie server
3.1 环境准备
(1)将文件移动到/user/local 目录
[root@cm-server-1 local]# cp oozie-5.0.0/distro/target/oozie-5.0.0-distro.tar.gz /user/local
(2)删除/user/local 目录下的oozie-5.0.0文件，因为解压oozie-5.0.0-distro.tar.gz文件解压后的文件名也是oozie-5.0.0，所以为防止文件冲突，将/user/local下的oozie-5.0.0文件删除
[root@cm-server-1 local]# rm -rf oozie-5.0.0
（3）将ExtJS 2.2的zip包添加到文件夹libext中（如果不需要要oozie的web功能可以不添加） 可以直接下载后复制到libext文件夹中，亦可以用如下wget命令下载
[root@cm-server-1 local]# wget http://archive.cloudera.com/gplextras/misc/ext-2.2.zip
（4）将hadoop对应版本下的lib目录下的文件复制libext
下载对于版本的hadoop，本文是2.6.0
[root@cm-server-1 local]# wget https://archive.apache.org/dist/hadoop/common/hadoop-2.6.0/hadoop-2.6.0.tar.gz
3.2 为oozie程序配置hadoop集群的代理用户
在 Hadoop的core-site.xml中添加如下两项配置
官网给的参考
<!-- OOZIE -->
<property>
<name>hadoop.proxyuser.[OOZIE_SERVER_USER].hosts</name>
<value>[OOZIE_SERVER_HOSTNAME]</value>
</property>
<property>
<name>hadoop.proxyuser.[OOZIE_SERVER_USER].groups</name>
<value>[USER_GROUPS_THAT_ALLOW_IMPERSONATION]</value>
</property>
其中，[OOZIE_SERVER_USER]需要改为后面启动oozie的用户,如设置为oozie 我们可以配置为如下：
<property>
<name>hadoop.proxyuser.oozie.hosts</name>
<value>*</value>
</property>
<property>
<name>hadoop.proxyuser.oozie.groups</name>
<value>*</value>
</property>
不重启hadoop集群，而使配置生效
hdfs dfsadmin -refreshSuperUserGroupsConfiguration
yarn rmadmin -refreshSuperUserGroupsConfiguration
注：因为我们部署的系统中已经配置了这两项且[OOZIE_SERVER_USER]设置为oozie了，因此设置代理用户这一步就省了了
3.3 解压包
首先修改hadoop以及oozie压缩包的权限（这个好像有点多余）
[root@cm-server-1 local]# chmod 777 hadoop-2.6.0.tar.gz
[root@cm-server-1 local]# chmod 777 oozie-5.0.0-distro.tar.gz
(1) 解压oozie-5.0.0-distro.tar.gz到/user/local 目录下
[root@cm-server-1 local]# tar zxvf oozie-5.0.0-distro.tar.gz
解压后的目录结构如下：
[root@cm-server-1 local]# ll oozie-5.0.0
total 335732
drwxr-xr-x 2 root root 4096 Jul 25 17:25 bin
drwxr-xr-x 4 502 wheel 4096 Jul 25 17:25 conf
-rwsrwsrwt 1 root root 8807259 Jul 25 16:31 docs.zip
drwxr-xr-x 4 root root 4096 Jul 25 17:25 embedded-oozie-server
drwxr-xr-x 2 root root 4096 Jul 25 17:25 libtools
-rw-r--r-- 1 502 wheel 37664 Mar 27 20:22 LICENSE.txt
-rw-r--r-- 1 502 wheel 450 Mar 27 20:22 NOTICE.txt
-rwsrwsrwt 1 root root 9477951 Jul 25 16:30 oozie-client-5.0.0.tar.gz
drwxr-xr-x 2 root root 4096 Jul 25 17:25 oozie-core
-rwsrwsrwt 1 root root 47569 Jul 25 16:31 oozie-examples.tar.gz
-rwsrwsrwt 1 root root 325239739 Jul 25 16:31 oozie-sharelib-5.0.0.tar.gz
-rw-r--r-- 1 502 wheel 3084 Mar 27 20:22 README.txt
-rw-r--r-- 1 502 wheel 137736 Mar 27 20:22 release-log.txt
[root@cm-server-1 local]#
（2）解压hadoop到/user/local 目录下 解压hadoop，官网有这么一句
Expand the Hadoop distribution tar.gz (as the Oozie Unix user).
应该是解压hadoop的用户与运行oozie服务的用户想同吧，因为我们规划用oozie这个用户来部署运行oozie服务，所以处解压也用oozie用户吧(我还是用的root，因为我的oozie没有写local目录的权限，直接用root也没什么可以安装)
[root@cm-server-1 local]# tar zxvf hadoop-2.6.0.tar.gz
3.4 添加libext
官网
Create a libext/ directory in the directory where Oozie was expanded.
If using the ExtJS library copy the ZIP file to the libext/ directory. If hadoop and hcatalog libraries are not already included in the war, add the corresponding libraries to libext/ directory.
按照官网的说明进行如下操作
修改oozie-5.0.0文件夹权限（oozie用户操作需要些权限，embedded-oozie-server文件夹主要是启动时写入pid文件）
[root@cm-server-1 local]# chmod 777 oozie-5.0.0
[root@cm-server-1 local]# chmod 777 oozie-5.0.0/embedded-oozie-server/
在解压的文件夹下面添加libext文件夹
[root@cm-server-1 local]$ mkdir oozie-5.0.0/libext
(1) 添加ext2.2
[root@cm-server-1 local]# cp ext-2.2.zip oozie-5.0.0/libext/
(2)添加hadoop lib下的包
[root@cm-server-1 local]# cp hadoop-2.6.0/share/hadoop/*/*.jar oozie-5.0.0/libext/
[root@cm-server-1 local]# cp hadoop-2.6.0/share/hadoop/*/lib/*.jar oozie-5.0.0/libext/
（3）添加对于数据库的驱动
下载对应的驱动到libext文件夹，本文是postgresql，自带的不用下载了
3.5配置参数
(1)之后再在oozie-site.xml中添加如下代码
<property>
<name>oozie.service.JPAService.jdbc.driver</name>
<value>org.postgresql.Driver</value>
<description>
JDBC driver class.
</description>
</property>
<property>
<name>oozie.service.JPAService.jdbc.url</name>
<value>jdbc:postgresql://*.*.*.*:5432/oozie?createDatabaseIfNotExist=true</value>
<description>
JDBC URL.
</description>
</property>
<property>
<name>oozie.service.JPAService.jdbc.username</name>
<value>admin</value>
<description>
DB user name.
</description>
</property>
<property>
<name>oozie.service.JPAService.jdbc.password</name>
<value>123456</value>
<description>
DB user password.
</description>
</property>
<property>
<name>oozie.service.HadoopAccessorService.hadoop.configurations</name>
<value>*=/etc/hadoop/conf</value>
<description>
Comma separated AUTHORITY=HADOOP_CONF_DIR, where AUTHORITY is the HOST:PORT of
the Hadoop service (JobTracker, YARN, HDFS). The wildcard '*' configuration is
used when there is no exact match for an authority. The HADOOP_CONF_DIR contains
the relevant Hadoop *-site.xml files. If the path is relative is looked within
the Oozie configuration directory; though the path can be absolute (i.e. to point
to Hadoop client conf/ directories in the local filesystem.
</description>
</property>
<property>
<name>oozie.service.HadoopAccessorService.action.configurations</name>
<value>*=/etc/hadoop/conf</value>
<description>
Comma separated AUTHORITY=ACTION_CONF_DIR, where AUTHORITY is the HOST:PORT of
the Hadoop MapReduce service (JobTracker, YARN). The wildcard '*' configuration is
used when there is no exact match for an authority. The ACTION_CONF_DIR may contain
ACTION.xml files where ACTION is the action type ('java', 'map-reduce', 'pig',
'hive', 'sqoop', etc.). If the ACTION.xml file exists, its properties will be used
as defaults properties for the action. If the path is relative is looked within
the Oozie configuration directory; though the path can be absolute (i.e. to point
to Hadoop client conf/ directories in the local filesystem.
</description>
</property>
<property>
<name>oozie.service.SparkConfigurationService.spark.configurations</name>
<value>*=/etc/spark2/conf</value>
<description>
Comma separated AUTHORITY=SPARK_CONF_DIR, where AUTHORITY is the HOST:PORT of
the ResourceManager of a YARN cluster. The wildcard '*' configuration is
used when there is no exact match for an authority. The SPARK_CONF_DIR contains
the relevant spark-defaults.conf properties file. If the path is relative is looked within
the Oozie configuration directory; though the path can be absolute. This is only used
when the Spark master is set to either "yarn-client" or "yarn-cluster".
</description>
</property>
(2)运行ooziedb.sh命令行工具创建数据库
[root@cm-server-1 oozie-5.0.0]# bin/ooziedb.sh create -sqlfile oozie.sql -run
运行完命令之后会在数据库中生成表，并在本目录下生成oozie.sql文件
3.5 上传jar到hdfs
上传sharelib到hdfs中，这里上传的jar包都是用户程序通过oozie运行时用到的。而libext中添加的hadoopjar是oozie系统本身用到的，oozie相当于hadoop的一个调用客户端
[oozie@cm-server-1 oozie-5.0.0]$ sudo -u oozie bin/oozie-setup.sh sharelib create -fs hdfs://cm-server-1.pm.com:8020 -locallib oozie-sharelib-5.0.0.tar.gz
执行完之后再hafs中产生如下/ user/ oozie/ share/ lib/ lib_20180726163837 目录
4 运行
4.1 环境准备
[root@cm-server-1 oozie-5.0.0]# sudo -u oozie bin/oozie-setup.sh
运行成功之后会出现INFO: Oozie is ready to be started的信息
这一步会做一些运行前准备工作，将libext中的文件copy到jetty的webapp/WEB-INF/lib下的lib目录中
注意：运行成功之后webapp/WEB-INF/lib下的jar会有冲突，需要将从hadoop添加的包手动删除
4.2 启动
sudo -u oozie bin/oozied.sh start
4.3 测试
bin/oozie admin -oozie http://localhost:11000/oozie -status
输出System mode: NORMAL,表示系统正常
参考文献
http://oozie.apache.org/docs/5.0.0/DG_QuickStart.html
https://www.cnblogs.com/juncaoit/p/6122187.html
转载于:https://my.oschina.net/u/3825598/blog/1865904
最后
以上就是活力大雁为你收集整理的oozie-5.0.0安装部署1.下载oozie5.02.编译oozie3. 安装oozie server4 运行参考文献的全部内容，希望文章能够帮你解决oozie-5.0.0安装部署1.下载oozie5.02.编译oozie3. 安装oozie server4 运行参考文献所遇到的程序开发问题。
如果觉得靠谱客网站的内容还不错，欢迎将靠谱客网站推荐给程序员好友。
发表评论 取消回复