概述
Hadoop生态圈-Oozie实战之调度shell脚本
作者：尹正杰
版权声明：原创作品，谢绝转载！否则将追究法律责任。
本篇博客展示案例：使用Oozie调度Shell脚本。
1>.解压官方案例模板
[root@s101 oozie-4.0.0-cdh5.3.6]# ll total 553892 drwxr-xr-x 2 1106 4001 4096 Jul 28 2015 bin drwxr-xr-x 4 1106 4001 4096 Sep 6 21:08 conf drwxr-xr-x 6 1106 4001 4096 Jul 28 2015 docs drwxr-xr-x 4 1106 4001 114 Jul 28 2015 hadooplibs drwxr-xr-x 2 1106 4001 4096 Jul 28 2015 lib drwxr-xr-x 2 root root 4096 Sep 6 21:05 libext drwxr-xr-x 2 1106 4001 8192 Jul 28 2015 libtools -rw-r--r-- 1 1106 4001 37664 Jul 28 2015 LICENSE.txt drwxr-xr-x 2 root root 4096 Sep 6 21:42 logs -rw-r--r-- 1 1106 4001 909 Jul 28 2015 NOTICE.txt drwxr-xr-x 2 1106 4001 84 Jul 28 2015 oozie-core -rw-r--r-- 1 1106 4001 42353 Jul 28 2015 oozie-examples.tar.gz -rw-r--r-- 1 1106 4001 80716120 Jul 28 2015 oozie-hadooplibs-4.0.0-cdh5.3.6.tar.gz drwxr-xr-x 9 1106 4001 149 Jul 28 2015 oozie-server -r--r--r-- 1 1106 4001 200450272 Jul 28 2015 oozie-sharelib-4.0.0-cdh5.3.6.tar.gz -r--r--r-- 1 1106 4001 200379680 Jul 28 2015 oozie-sharelib-4.0.0-cdh5.3.6-yarn.tar.gz -rw-r--r-- 1 root root 7562 Sep 6 21:33 oozie.sql -rw-r--r-- 1 1106 4001 85427394 Jul 28 2015 oozie.war -rw-r--r-- 1 1106 4001 58910 Jul 28 2015 release-log.txt drwxr-xr-x 20 1106 4001 4096 Jul 28 2015 src [root@s101 oozie-4.0.0-cdh5.3.6]# [root@s101 oozie-4.0.0-cdh5.3.6]# tar -zxf oozie-examples.tar.gz [root@s101 oozie-4.0.0-cdh5.3.6]# ll total 553892 drwxr-xr-x 2 1106 4001 4096 Jul 28 2015 bin drwxr-xr-x 4 1106 4001 4096 Sep 6 21:08 conf drwxr-xr-x 6 1106 4001 4096 Jul 28 2015 docs drwxr-xr-x 5 root root 44 Sep 6 22:08 examples drwxr-xr-x 4 1106 4001 114 Jul 28 2015 hadooplibs drwxr-xr-x 2 1106 4001 4096 Jul 28 2015 lib drwxr-xr-x 2 root root 4096 Sep 6 21:05 libext drwxr-xr-x 2 1106 4001 8192 Jul 28 2015 libtools -rw-r--r-- 1 1106 4001 37664 Jul 28 2015 LICENSE.txt drwxr-xr-x 2 root root 4096 Sep 6 21:42 logs -rw-r--r-- 1 1106 4001 909 Jul 28 2015 NOTICE.txt drwxr-xr-x 2 1106 4001 84 Jul 28 2015 oozie-core -rw-r--r-- 1 1106 4001 42353 Jul 28 2015 oozie-examples.tar.gz -rw-r--r-- 1 1106 4001 80716120 Jul 28 2015 oozie-hadooplibs-4.0.0-cdh5.3.6.tar.gz drwxr-xr-x 9 1106 4001 149 Jul 28 2015 oozie-server -r--r--r-- 1 1106 4001 200450272 Jul 28 2015 oozie-sharelib-4.0.0-cdh5.3.6.tar.gz -r--r--r-- 1 1106 4001 200379680 Jul 28 2015 oozie-sharelib-4.0.0-cdh5.3.6-yarn.tar.gz -rw-r--r-- 1 root root 7562 Sep 6 21:33 oozie.sql -rw-r--r-- 1 1106 4001 85427394 Jul 28 2015 oozie.war -rw-r--r-- 1 1106 4001 58910 Jul 28 2015 release-log.txt drwxr-xr-x 20 1106 4001 4096 Jul 28 2015 src [root@s101 oozie-4.0.0-cdh5.3.6]# [root@s101 oozie-4.0.0-cdh5.3.6]# [root@s101 oozie-4.0.0-cdh5.3.6]# ll | grep examples drwxr-xr-x 5 root root 44 Sep 6 22:08 examples -rw-r--r-- 1 1106 4001 42353 Jul 28 2015 oozie-examples.tar.gz [root@s101 oozie-4.0.0-cdh5.3.6]#
2>. 创建工作目录
[root@s101 oozie-4.0.0-cdh5.3.6]# mkdir yinzhengjie-oozie-jobs [root@s101 oozie-4.0.0-cdh5.3.6]#
3>.拷贝任务模板到oozie-apps/目录
[root@s101 oozie-4.0.0-cdh5.3.6]# cp -r examples/apps/shell/ yinzhengjie-oozie-jobs/ [root@s101 oozie-4.0.0-cdh5.3.6]#
4>. 随意编写一个脚本yinzhengjie.sh
[root@yinzhengjie oozie-4.0.0-cdh5.3.6]# vi yinzhengjie-oozie-jobs/shell/yinzhengjie.sh [root@yinzhengjie oozie-4.0.0-cdh5.3.6]# [root@yinzhengjie oozie-4.0.0-cdh5.3.6]# more yinzhengjie-oozie-jobs/shell/yinzhengjie.sh #!/bin/bash #@author :yinzhengjie #blog:http://www.cnblogs.com/yinzhengjie #EMAIL:y1053419035@qq.com function yinzhengjie(){ currentTime=`date -d today +"%Y-%m-%d %T"` echo " $currentTime : 尹正杰到此一游！" >>/home/yinzhengjie/data/access.log } yinzhengjie [root@yinzhengjie oozie-4.0.0-cdh5.3.6]# [root@yinzhengjie oozie-4.0.0-cdh5.3.6]#
5>.修改job.properties
[root@yinzhengjie oozie-4.0.0-cdh5.3.6]# vi yinzhengjie-oozie-jobs/shell/job.properties [root@yinzhengjie oozie-4.0.0-cdh5.3.6]# [root@yinzhengjie oozie-4.0.0-cdh5.3.6]# more yinzhengjie-oozie-jobs/shell/job.properties # # Licensed to the Apache Software Foundation (ASF) under one # or more contributor license agreements. See the NOTICE file # distributed with this work for additional information # regarding copyright ownership. The ASF licenses this file # to you under the Apache License, Version 2.0 (the # "License"); you may not use this file except in compliance # with the License. You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. # #HDFS地址 nameNode=hdfs://s101:8020 #ResourceManager地址 jobTracker=s101:8021 #队列名称 queueName=default examplesRoot=yinzhengjie-oozie-jobs #指定oozie的shell脚本存放路径 oozie.wf.application.path=${nameNode}/user/${user.name}/${examplesRoot}/shell #指定执行的脚本名称 EXEC=yinzhengjie.sh [root@yinzhengjie oozie-4.0.0-cdh5.3.6]# [root@yinzhengjie oozie-4.0.0-cdh5.3.6]#
6>.修改workflow.xml文件
1 [root@yinzhengjie oozie-4.0.0-cdh5.3.6]# vi yinzhengjie-oozie-jobs/shell/workflow.xml 2 [root@yinzhengjie oozie-4.0.0-cdh5.3.6]# 3 [root@yinzhengjie oozie-4.0.0-cdh5.3.6]# 4 [root@yinzhengjie oozie-4.0.0-cdh5.3.6]# 5 [root@yinzhengjie oozie-4.0.0-cdh5.3.6]# more yinzhengjie-oozie-jobs/shell/workflow.xml 6 <workflow-app xmlns="uri:oozie:workflow:0.4" name="shell-wf"> 7 8 <!-- 任务开始会先执行start标签，而start标签指定是要执行的动作，即action。使用name属性指定action的名称 --> 9 <start to="shell-node"/> 10 11 <!-- action是定义执行的动作，使用name属性定义action的名称 --> 12 <action name="shell-node"> 13 14 <!-- 由于我们指定的是linux的shell脚本，因此这里使用shell标签包裹--> 15 <shell xmlns="uri:oozie:shell-action:0.2"> 16 17 18 <!-- 注意，这里的“${jobTracker}”变量是调用的job.properties配置文件中的变量哟，下面调用变量同理。--> 19 <job-tracker>${jobTracker}</job-tracker> 20 <name-node>${nameNode}</name-node> 21 <configuration> 22 23 <!--指定队列的名称--> 24 <property> 25 <name>mapred.job.queue.name</name> 26 <value>${queueName}</value> 27 </property> 28 </configuration> 29 30 <!-- 指定执行脚本的名称--> 31 <exec>${EXEC}</exec> 32 33 <!-- 注意，argument关键字这一行是传参数，我们可以传递多个参数，由于我自定义的脚本并没有传递参数的需求，因此这里就不是必须的，当前行是注释掉的。--> 34 <!-- <argument>my_output=Hello Oozie</argument> --> 35 36 <!-- 指定hdfs执行脚本的路径--> 37 <file>/user/root/yinzhengjie-oozie-jobs/shell/${EXEC}#${EXEC}</file> 38 39 <capture-output/> 40 </shell> 41 42 <!-- 当执行成功时会执行的标签，即会执行下面的“end标签”--> 43 <ok to="end"/> 44 45 <!-- 当执行出错时会执行的标签,即会执行下面的“fail标签”--> 46 <error to="fail"/> 47 </action> 48 49 50 <!-- 定义kill标签，其名称为“fail”--> 51 <kill name="fail"> 52 <message>Shell action failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message> 53 </kill> 54 55 <!-- 定义执行“end”标签，其名称为“end”--> 56 <end name="end"/> 57 58 </workflow-app> 59 [root@yinzhengjie oozie-4.0.0-cdh5.3.6]#
7>.上传任务配置
[root@s101 oozie-4.0.0-cdh5.3.6]# /home/yinzhengjie/download/cdh/hadoop-2.5.0-cdh5.3.6/bin/hdfs dfs -put yinzhengjie-oozie-jobs/ /user/root [root@s101 oozie-4.0.0-cdh5.3.6]# /home/yinzhengjie/download/cdh/hadoop-2.5.0-cdh5.3.6/bin/hdfs dfs -ls -R /user/root/ drwxr-xr-x - root supergroup 0 2018-09-06 21:27 /user/root/share drwxr-xr-x - root supergroup 0 2018-09-06 21:27 /user/root/share/lib drwxr-xr-x - root supergroup 0 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707 drwxr-xr-x - root supergroup 0 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/distcp -rw-r--r-- 1 root supergroup 98436 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/distcp/hadoop-distcp-2.5.0-cdh5.3.6.jar -rw-r--r-- 1 root supergroup 7586 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/distcp/oozie-sharelib-distcp-4.0.0-cdh5.3.6.jar drwxr-xr-x - root supergroup 0 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hcatalog -rw-r--r-- 1 root supergroup 236660 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hcatalog/ST4-4.0.4.jar -rw-r--r-- 1 root supergroup 1514270 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hcatalog/ant-1.8.1.jar -rw-r--r-- 1 root supergroup 12312 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hcatalog/ant-launcher-1.8.1.jar -rw-r--r-- 1 root supergroup 437647 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hcatalog/avro-1.7.6-cdh5.3.6.jar -rw-r--r-- 1 root supergroup 115709 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hcatalog/bonecp-0.7.1.RELEASE.jar -rw-r--r-- 1 root supergroup 241367 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hcatalog/commons-compress-1.4.1.jar -rw-r--r-- 1 root supergroup 305001 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hcatalog/commons-httpclient-3.1.jar -rw-r--r-- 1 root supergroup 185140 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hcatalog/commons-io-2.4.jar -rw-r--r-- 1 root supergroup 315805 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hcatalog/commons-lang3-3.1.jar -rw-r--r-- 1 root supergroup 337012 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hcatalog/datanucleus-api-jdo-3.2.1.jar -rw-r--r-- 1 root supergroup 1801810 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hcatalog/datanucleus-core-3.2.2.jar -rw-r--r-- 1 root supergroup 1769726 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hcatalog/datanucleus-rdbms-3.2.1.jar -rw-r--r-- 1 root supergroup 6377448 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hcatalog/groovy-all-2.1.6.jar -rw-r--r-- 1 root supergroup 37256 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hcatalog/hive-ant-0.13.1-cdh5.3.6.jar -rw-r--r-- 1 root supergroup 187151 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hcatalog/hive-common-0.13.1-cdh5.3.6.jar -rw-r--r-- 1 root supergroup 15678549 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hcatalog/hive-exec-0.13.1-cdh5.3.6.jar -rw-r--r-- 1 root supergroup 467908 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hcatalog/hive-hcatalog-core-0.13.1-cdh5.3.6.jar -rw-r--r-- 1 root supergroup 80969 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hcatalog/hive-hcatalog-pig-adapter-0.13.1-cdh5.3.6.jar -rw-r--r-- 1 root supergroup 67747 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hcatalog/hive-hcatalog-server-extensions-0.13.1-cdh5.3.6.jar -rw-r--r-- 1 root supergroup 4695460 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hcatalog/hive-metastore-0.13.1-cdh5.3.6.jar -rw-r--r-- 1 root supergroup 767934 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hcatalog/hive-serde-0.13.1-cdh5.3.6.jar -rw-r--r-- 1 root supergroup 79054 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hcatalog/hive-webhcat-java-client-0.13.1-cdh5.3.6.jar -rw-r--r-- 1 root supergroup 201124 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hcatalog/jdo-api-3.0.1.jar -rw-r--r-- 1 root supergroup 458739 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hcatalog/jersey-core-1.9.jar -rw-r--r-- 1 root supergroup 713089 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hcatalog/jersey-server-1.9.jar -rw-r--r-- 1 root supergroup 125573 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hcatalog/jersey-servlet-1.14.jar -rw-r--r-- 1 root supergroup 275186 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hcatalog/libfb303-0.9.0.jar -rw-r--r-- 1 root supergroup 481535 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hcatalog/log4j-1.2.16.jar -rw-r--r-- 1 root supergroup 9275 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hcatalog/oozie-hcatalog-0.13.1-cdh5.3.6.oozie-4.0.0-cdh5.3.6.jar -rw-r--r-- 1 root supergroup 18120 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hcatalog/oozie-sharelib-hcatalog-4.0.0-cdh5.3.6.jar -rw-r--r-- 1 root supergroup 19827 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hcatalog/opencsv-2.3.jar -rw-r--r-- 1 root supergroup 29555 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hcatalog/paranamer-2.3.jar -rw-r--r-- 1 root supergroup 26084 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hcatalog/slf4j-api-1.7.5.jar -rw-r--r-- 1 root supergroup 8869 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hcatalog/slf4j-log4j12-1.7.5.jar -rw-r--r-- 1 root supergroup 995968 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hcatalog/snappy-java-1.0.4.1.jar -rw-r--r-- 1 root supergroup 26514 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hcatalog/stax-api-1.0.1.jar -rw-r--r-- 1 root supergroup 449505 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hcatalog/velocity-1.7.jar -rw-r--r-- 1 root supergroup 94672 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hcatalog/xz-1.0.jar drwxr-xr-x - root supergroup 0 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive -rw-r--r-- 1 root supergroup 236660 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive/ST4-4.0.4.jar -rw-r--r-- 1 root supergroup 62983 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive/activation-1.1.jar -rw-r--r-- 1 root supergroup 1514270 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive/ant-1.8.1.jar -rw-r--r-- 1 root supergroup 12312 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive/ant-launcher-1.8.1.jar -rw-r--r-- 1 root supergroup 445288 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive/antlr-2.7.7.jar -rw-r--r-- 1 root supergroup 164368 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive/antlr-runtime-3.4.jar -rw-r--r-- 1 root supergroup 32693 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive/asm-commons-3.1.jar -rw-r--r-- 1 root supergroup 21879 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive/asm-tree-3.1.jar -rw-r--r-- 1 root supergroup 115709 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive/bonecp-0.7.1.RELEASE.jar -rw-r--r-- 1 root supergroup 241367 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive/commons-compress-1.4.1.jar -rw-r--r-- 1 root supergroup 305001 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive/commons-httpclient-3.1.jar -rw-r--r-- 1 root supergroup 185140 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive/commons-io-2.4.jar -rw-r--r-- 1 root supergroup 315805 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive/commons-lang3-3.1.jar -rw-r--r-- 1 root supergroup 337012 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive/datanucleus-api-jdo-3.2.1.jar -rw-r--r-- 1 root supergroup 1801810 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive/datanucleus-core-3.2.2.jar -rw-r--r-- 1 root supergroup 1769726 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive/datanucleus-rdbms-3.2.1.jar -rw-r--r-- 1 root supergroup 2831358 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive/derby-10.10.1.1.jar -rw-r--r-- 1 root supergroup 12452 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive/geronimo-annotation_1.0_spec-1.1.1.jar -rw-r--r-- 1 root supergroup 30548 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive/geronimo-jaspic_1.0_spec-1.0.jar -rw-r--r-- 1 root supergroup 16030 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive/geronimo-jta_1.1_spec-1.1.1.jar -rw-r--r-- 1 root supergroup 6377448 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive/groovy-all-2.1.6.jar -rw-r--r-- 1 root supergroup 1648200 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive/guava-11.0.2.jar -rw-r--r-- 1 root supergroup 37256 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive/hive-ant-0.13.1-cdh5.3.6.jar -rw-r--r-- 1 root supergroup 37314 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive/hive-cli-0.13.1-cdh5.3.6.jar -rw-r--r-- 1 root supergroup 187151 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive/hive-common-0.13.1-cdh5.3.6.jar -rw-r--r-- 1 root supergroup 121507 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive/hive-contrib-0.13.1-cdh5.3.6.jar -rw-r--r-- 1 root supergroup 15678549 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive/hive-exec-0.13.1-cdh5.3.6.jar -rw-r--r-- 1 root supergroup 4695460 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive/hive-metastore-0.13.1-cdh5.3.6.jar -rw-r--r-- 1 root supergroup 767934 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive/hive-serde-0.13.1-cdh5.3.6.jar -rw-r--r-- 1 root supergroup 1908940 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive/hive-service-0.13.1-cdh5.3.6.jar -rw-r--r-- 1 root supergroup 7424 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive/hive-shims-0.13.1-cdh5.3.6.jar -rw-r--r-- 1 root supergroup 49468 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive/hive-shims-0.23-0.13.1-cdh5.3.6.jar -rw-r--r-- 1 root supergroup 43371 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive/hive-shims-common-0.13.1-cdh5.3.6.jar -rw-r--r-- 1 root supergroup 78719 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive/hive-shims-common-secure-0.13.1-cdh5.3.6.jar -rw-r--r-- 1 root supergroup 12990 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive/hive-shims-scheduler-0.13.1-cdh5.3.6.jar -rw-r--r-- 1 root supergroup 433368 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive/httpclient-4.2.5.jar -rw-r--r-- 1 root supergroup 227708 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive/httpcore-4.2.5.jar -rw-r--r-- 1 root supergroup 201124 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive/jdo-api-3.0.1.jar -rw-r--r-- 1 root supergroup 458739 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive/jersey-core-1.9.jar -rw-r--r-- 1 root supergroup 713089 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive/jersey-server-1.9.jar -rw-r--r-- 1 root supergroup 125573 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive/jersey-servlet-1.14.jar -rw-r--r-- 1 root supergroup 1681148 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive/jetty-all-7.6.0.v20120127.jar -rw-r--r-- 1 root supergroup 87325 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive/jline-0.9.94.jar -rw-r--r-- 1 root supergroup 12131 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive/jpam-1.1.jar -rw-r--r-- 1 root supergroup 33015 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive/jsr305-1.3.9.jar -rw-r--r-- 1 root supergroup 15071 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive/jta-1.1.jar -rw-r--r-- 1 root supergroup 275186 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive/libfb303-0.9.0.jar -rw-r--r-- 1 root supergroup 348651 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive/libthrift-0.9.0-cdh5-2.jar -rw-r--r-- 1 root supergroup 481535 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive/log4j-1.2.16.jar -rw-r--r-- 1 root supergroup 388864 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive/mail-1.4.jar -rw-r--r-- 1 root supergroup 17253 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive/oozie-sharelib-hive-4.0.0-cdh5.3.6.jar -rw-r--r-- 1 root supergroup 19827 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive/opencsv-2.3.jar -rw-r--r-- 1 root supergroup 26084 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive/slf4j-api-1.7.5.jar -rw-r--r-- 1 root supergroup 8869 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive/slf4j-log4j12-1.7.5.jar -rw-r--r-- 1 root supergroup 26514 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive/stax-api-1.0.1.jar -rw-r--r-- 1 root supergroup 148627 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive/stringtemplate-3.2.1.jar -rw-r--r-- 1 root supergroup 94672 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive/xz-1.0.jar drwxr-xr-x - root supergroup 0 2018-09-06 21:28 /user/root/share/lib/lib_20180906212707/hive2 -rw-r--r-- 1 root supergroup 236660 2018-09-06 21:28 /user/root/share/lib/lib_20180906212707/hive2/ST4-4.0.4.jar -rw-r--r-- 1 root supergroup 62983 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive2/activation-1.1.jar -rw-r--r-- 1 root supergroup 445288 2018-09-06 21:28 /user/root/share/lib/lib_20180906212707/hive2/antlr-2.7.7.jar -rw-r--r-- 1 root supergroup 164368 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive2/antlr-runtime-3.4.jar -rw-r--r-- 1 root supergroup 43398 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive2/asm-3.2.jar -rw-r--r-- 1 root supergroup 32693 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive2/asm-commons-3.1.jar -rw-r--r-- 1 root supergroup 21879 2018-09-06 21:28 /user/root/share/lib/lib_20180906212707/hive2/asm-tree-3.1.jar -rw-r--r-- 1 root supergroup 41123 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive2/commons-cli-1.2.jar -rw-r--r-- 1 root supergroup 58160 2018-09-06 21:28 /user/root/share/lib/lib_20180906212707/hive2/commons-codec-1.4.jar -rw-r--r-- 1 root supergroup 305001 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive2/commons-httpclient-3.1.jar -rw-r--r-- 1 root supergroup 185140 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive2/commons-io-2.4.jar -rw-r--r-- 1 root supergroup 261809 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive2/commons-lang-2.4.jar -rw-r--r-- 1 root supergroup 315805 2018-09-06 21:28 /user/root/share/lib/lib_20180906212707/hive2/commons-lang3-3.1.jar -rw-r--r-- 1 root supergroup 52915 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive2/commons-logging-1.1.jar -rw-r--r-- 1 root supergroup 337012 2018-09-06 21:28 /user/root/share/lib/lib_20180906212707/hive2/datanucleus-api-jdo-3.2.1.jar -rw-r--r-- 1 root supergroup 1801810 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive2/datanucleus-core-3.2.2.jar -rw-r--r-- 1 root supergroup 1769726 2018-09-06 21:28 /user/root/share/lib/lib_20180906212707/hive2/datanucleus-rdbms-3.2.1.jar -rw-r--r-- 1 root supergroup 12452 2018-09-06 21:28 /user/root/share/lib/lib_20180906212707/hive2/geronimo-annotation_1.0_spec-1.1.1.jar -rw-r--r-- 1 root supergroup 30548 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive2/geronimo-jaspic_1.0_spec-1.0.jar -rw-r--r-- 1 root supergroup 16030 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive2/geronimo-jta_1.1_spec-1.1.1.jar -rw-r--r-- 1 root supergroup 122902 2018-09-06 21:28 /user/root/share/lib/lib_20180906212707/hive2/hive-beeline-0.13.1-cdh5.3.6.jar -rw-r--r-- 1 root supergroup 15678549 2018-09-06 21:28 /user/root/share/lib/lib_20180906212707/hive2/hive-exec-0.13.1-cdh5.3.6.jar -rw-r--r-- 1 root supergroup 136904 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive2/hive-jdbc-0.13.1-cdh5.3.6.jar -rw-r--r-- 1 root supergroup 4695460 2018-09-06 21:28 /user/root/share/lib/lib_20180906212707/hive2/hive-metastore-0.13.1-cdh5.3.6.jar -rw-r--r-- 1 root supergroup 767934 2018-09-06 21:28 /user/root/share/lib/lib_20180906212707/hive2/hive-serde-0.13.1-cdh5.3.6.jar -rw-r--r-- 1 root supergroup 1908940 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive2/hive-service-0.13.1-cdh5.3.6.jar -rw-r--r-- 1 root supergroup 7424 2018-09-06 21:28 /user/root/share/lib/lib_20180906212707/hive2/hive-shims-0.13.1-cdh5.3.6.jar -rw-r--r-- 1 root supergroup 49468 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive2/hive-shims-0.23-0.13.1-cdh5.3.6.jar -rw-r--r-- 1 root supergroup 43371 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive2/hive-shims-common-0.13.1-cdh5.3.6.jar -rw-r--r-- 1 root supergroup 78719 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive2/hive-shims-common-secure-0.13.1-cdh5.3.6.jar -rw-r--r-- 1 root supergroup 12990 2018-09-06 21:28 /user/root/share/lib/lib_20180906212707/hive2/hive-shims-scheduler-0.13.1-cdh5.3.6.jar -rw-r--r-- 1 root supergroup 433368 2018-09-06 21:28 /user/root/share/lib/lib_20180906212707/hive2/httpclient-4.2.5.jar -rw-r--r-- 1 root supergroup 227708 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive2/httpcore-4.2.5.jar -rw-r--r-- 1 root supergroup 201124 2018-09-06 21:28 /user/root/share/lib/lib_20180906212707/hive2/jdo-api-3.0.1.jar -rw-r--r-- 1 root supergroup 458739 2018-09-06 21:28 /user/root/share/lib/lib_20180906212707/hive2/jersey-core-1.9.jar -rw-r--r-- 1 root supergroup 713089 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive2/jersey-server-1.9.jar -rw-r--r-- 1 root supergroup 125573 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive2/jersey-servlet-1.14.jar -rw-r--r-- 1 root supergroup 1681148 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive2/jetty-all-7.6.0.v20120127.jar -rw-r--r-- 1 root supergroup 87325 2018-09-06 21:28 /user/root/share/lib/lib_20180906212707/hive2/jline-0.9.94.jar -rw-r--r-- 1 root supergroup 12131 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive2/jpam-1.1.jar -rw-r--r-- 1 root supergroup 15071 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive2/jta-1.1.jar -rw-r--r-- 1 root supergroup 275186 2018-09-06 21:28 /user/root/share/lib/lib_20180906212707/hive2/libfb303-0.9.0.jar -rw-r--r-- 1 root supergroup 348651 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/hive2/libthrift-0.9.0-cdh5-2.jar -rw-r--r-- 1 root supergroup 481535 2018-09-06 21:28 /user/root/share/lib/lib_20180906212707/hive2/log4j-1.2.16.jar -rw-r--r-- 1 root supergroup 388864 2018-09-06 21:28 /user/root/share/lib/lib_20180906212707/hive2/mail-1.4.jar -rw-r--r-- 1 root supergroup 21533 2018-09-06 21:28 /user/root/share/lib/lib_20180906212707/hive2/oozie-sharelib-hive2-4.0.0-cdh5.3.6.jar -rw-r--r-- 1 root supergroup 19827 2018-09-06 21:28 /user/root/share/lib/lib_20180906212707/hive2/opencsv-2.3.jar -rw-r--r-- 1 root supergroup 105112 2018-09-06 21:28 /user/root/share/lib/lib_20180906212707/hive2/servlet-api-2.5.jar -rw-r--r-- 1 root supergroup 26084 2018-09-06 21:28 /user/root/share/lib/lib_20180906212707/hive2/slf4j-api-1.7.5.jar -rw-r--r-- 1 root supergroup 8869 2018-09-06 21:28 /user/root/share/lib/lib_20180906212707/hive2/slf4j-log4j12-1.7.5.jar -rw-r--r-- 1 root supergroup 148627 2018-09-06 21:28 /user/root/share/lib/lib_20180906212707/hive2/stringtemplate-3.2.1.jar -rw-r--r-- 1 root supergroup 93210 2018-09-06 21:28 /user/root/share/lib/lib_20180906212707/hive2/super-csv-2.2.0.jar drwxr-xr-x - root supergroup 0 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/mapreduce-streaming -rw-r--r-- 1 root supergroup 109922 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/mapreduce-streaming/hadoop-streaming-2.5.0-cdh5.3.6.jar -rw-r--r-- 1 root supergroup 10339 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/mapreduce-streaming/oozie-sharelib-streaming-4.0.0-cdh5.3.6.jar drwxr-xr-x - root supergroup 0 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/oozie -rw-r--r-- 1 root supergroup 16046 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/oozie/json-simple-1.1.jar -rw-r--r-- 1 root supergroup 10138 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/oozie/oozie-hadoop-utils-2.5.0-cdh5.3.6.oozie-4.0.0-cdh5.3.6.jar -rw-r--r-- 1 root supergroup 47949 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/oozie/oozie-sharelib-oozie-4.0.0-cdh5.3.6.jar drwxr-xr-x - root supergroup 0 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/pig -rw-r--r-- 1 root supergroup 1034049 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/pig/ant-1.6.5.jar -rw-r--r-- 1 root supergroup 445288 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/pig/antlr-2.7.7.jar -rw-r--r-- 1 root supergroup 164368 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/pig/antlr-runtime-3.4.jar -rw-r--r-- 1 root supergroup 176285 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/pig/automaton-1.11-8.jar -rw-r--r-- 1 root supergroup 437647 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/pig/avro-1.7.6-cdh5.3.6.jar -rw-r--r-- 1 root supergroup 58160 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/pig/commons-codec-1.4.jar -rw-r--r-- 1 root supergroup 575389 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/pig/commons-collections-3.2.1.jar -rw-r--r-- 1 root supergroup 241367 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/pig/commons-compress-1.4.1.jar -rw-r--r-- 1 root supergroup 112341 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/pig/commons-el-1.0.jar -rw-r--r-- 1 root supergroup 305001 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/pig/commons-httpclient-3.1.jar -rw-r--r-- 1 root supergroup 185140 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/pig/commons-io-2.4.jar -rw-r--r-- 1 root supergroup 261809 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/pig/commons-lang-2.4.jar -rw-r--r-- 1 root supergroup 52915 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/pig/commons-logging-1.1.jar -rw-r--r-- 1 root supergroup 3566844 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/pig/core-3.1.1.jar -rw-r--r-- 1 root supergroup 15322 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/pig/findbugs-annotations-1.3.9-1.jar -rw-r--r-- 1 root supergroup 1648200 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/pig/guava-11.0.2.jar -rw-r--r-- 1 root supergroup 103836 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/pig/hbase-client-0.98.6-cdh5.3.6-tests.jar -rw-r--r-- 1 root supergroup 966708 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/pig/hbase-client-0.98.6-cdh5.3.6.jar -rw-r--r-- 1 root supergroup 451766 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/pig/hbase-common-0.98.6-cdh5.3.6.jar -rw-r--r-- 1 root supergroup 3554346 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/pig/hbase-protocol-0.98.6-cdh5.3.6.jar -rw-r--r-- 1 root supergroup 706710 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/pig/hsqldb-1.8.0.10.jar -rw-r--r-- 1 root supergroup 31532 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/pig/htrace-core-2.04.jar -rw-r--r-- 1 root supergroup 227500 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/pig/jackson-core-asl-1.8.8.jar -rw-r--r-- 1 root supergroup 668564 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/pig/jackson-mapper-asl-1.8.8.jar -rw-r--r-- 1 root supergroup 113796 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/pig/jansi-1.9.jar -rw-r--r-- 1 root supergroup 408133 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/pig/jasper-compiler-5.5.23.jar -rw-r--r-- 1 root supergroup 76844 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/pig/jasper-runtime-5.5.23.jar -rw-r--r-- 1 root supergroup 321806 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/pig/jets3t-0.6.1.jar -rw-r--r-- 1 root supergroup 516429 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/pig/jetty-6.1.14.jar -rw-r--r-- 1 root supergroup 177417 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/pig/jetty-util-6.1.26.cloudera.2.jar -rw-r--r-- 1 root supergroup 87325 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/pig/jline-0.9.94.jar -rw-r--r-- 1 root supergroup 534827 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/pig/joda-time-1.6.jar -rw-r--r-- 1 root supergroup 185746 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/pig/jsch-0.1.42.jar -rw-r--r-- 1 root supergroup 16046 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/pig/json-simple-1.1.jar -rw-r--r-- 1 root supergroup 1024680 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/pig/jsp-2.1-6.1.14.jar -rw-r--r-- 1 root supergroup 134910 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/pig/jsp-api-2.1-6.1.14.jar -rw-r--r-- 1 root supergroup 33015 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/pig/jsr305-1.3.9.jar -rw-r--r-- 1 root supergroup 14340135 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/pig/jython-standalone-2.5.3.jar -rw-r--r-- 1 root supergroup 11981 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/pig/kfs-0.3.jar -rw-r--r-- 1 root supergroup 481535 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/pig/log4j-1.2.16.jar -rw-r--r-- 1 root supergroup 1206119 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/pig/netty-3.6.6.Final.jar -rw-r--r-- 1 root supergroup 27640 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/pig/oozie-sharelib-pig-4.0.0-cdh5.3.6.jar -rw-r--r-- 1 root supergroup 65261 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/pig/oro-2.0.8.jar -rw-r--r-- 1 root supergroup 29555 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/pig/paranamer-2.3.jar -rw-r--r-- 1 root supergroup 2690376 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/pig/parquet-pig-bundle-1.5.0-cdh5.3.6.jar -rw-r--r-- 1 root supergroup 3798454 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/pig/pig-0.12.0-cdh5.3.6.jar -rw-r--r-- 1 root supergroup 378081 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/pig/piggybank-0.12.0-cdh5.3.6.jar -rw-r--r-- 1 root supergroup 533455 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/pig/protobuf-java-2.5.0.jar -rw-r--r-- 1 root supergroup 132368 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/pig/servlet-api-2.5-6.1.14.jar -rw-r--r-- 1 root supergroup 26084 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/pig/slf4j-api-1.7.5.jar -rw-r--r-- 1 root supergroup 995968 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/pig/snappy-java-1.0.4.1.jar -rw-r--r-- 1 root supergroup 148627 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/pig/stringtemplate-3.2.1.jar -rw-r--r-- 1 root supergroup 94672 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/pig/xz-1.0.jar -rw-r--r-- 1 root supergroup 1364 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/sharelib.properties drwxr-xr-x - root supergroup 0 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/sqoop -rw-r--r-- 1 root supergroup 236660 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/sqoop/ST4-4.0.4.jar -rw-r--r-- 1 root supergroup 62983 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/sqoop/activation-1.1.jar -rw-r--r-- 1 root supergroup 1514270 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/sqoop/ant-1.8.1.jar -rw-r--r-- 1 root supergroup 12312 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/sqoop/ant-launcher-1.8.1.jar -rw-r--r-- 1 root supergroup 445288 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/sqoop/antlr-2.7.7.jar -rw-r--r-- 1 root supergroup 164368 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/sqoop/antlr-runtime-3.4.jar -rw-r--r-- 1 root supergroup 43398 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/sqoop/asm-3.2.jar -rw-r--r-- 1 root supergroup 32693 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/sqoop/asm-commons-3.1.jar -rw-r--r-- 1 root supergroup 21879 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/sqoop/asm-tree-3.1.jar -rw-r--r-- 1 root supergroup 394013 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/sqoop/avro-ipc-1.7.6-cdh5.3.6-tests.jar -rw-r--r-- 1 root supergroup 129957 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/sqoop/avro-ipc-1.7.6-cdh5.3.6.jar -rw-r--r-- 1 root supergroup 180874 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/sqoop/avro-mapred-1.7.6-cdh5.3.6-hadoop2.jar -rw-r--r-- 1 root supergroup 115709 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/sqoop/bonecp-0.7.1.RELEASE.jar -rw-r--r-- 1 root supergroup 241367 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/sqoop/commons-compress-1.4.1.jar -rw-r--r-- 1 root supergroup 185140 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/sqoop/commons-io-2.4.jar -rw-r--r-- 1 root supergroup 267634 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/sqoop/commons-jexl-2.1.1.jar -rw-r--r-- 1 root supergroup 315805 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/sqoop/commons-lang3-3.1.jar -rw-r--r-- 1 root supergroup 337012 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/sqoop/datanucleus-api-jdo-3.2.1.jar -rw-r--r-- 1 root supergroup 1801810 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/sqoop/datanucleus-core-3.2.2.jar -rw-r--r-- 1 root supergroup 1769726 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/sqoop/datanucleus-rdbms-3.2.1.jar -rw-r--r-- 1 root supergroup 15322 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/sqoop/findbugs-annotations-1.3.9-1.jar -rw-r--r-- 1 root supergroup 12452 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/sqoop/geronimo-annotation_1.0_spec-1.1.1.jar -rw-r--r-- 1 root supergroup 30548 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/sqoop/geronimo-jaspic_1.0_spec-1.0.jar -rw-r--r-- 1 root supergroup 16030 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/sqoop/geronimo-jta_1.1_spec-1.1.1.jar -rw-r--r-- 1 root supergroup 6377448 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/sqoop/groovy-all-2.1.6.jar -rw-r--r-- 1 root supergroup 1648200 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/sqoop/guava-11.0.2.jar -rw-r--r-- 1 root supergroup 26151 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/sqoop/hadoop-archives-2.5.0-cdh5.3.6.jar -rw-r--r-- 1 root supergroup 451766 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/sqoop/hbase-common-0.98.6-cdh5.3.6.jar -rw-r--r-- 1 root supergroup 37256 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/sqoop/hive-ant-0.13.1-cdh5.3.6.jar -rw-r--r-- 1 root supergroup 37314 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/sqoop/hive-cli-0.13.1-cdh5.3.6.jar -rw-r--r-- 1 root supergroup 187151 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/sqoop/hive-common-0.13.1-cdh5.3.6.jar -rw-r--r-- 1 root supergroup 15678549 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/sqoop/hive-exec-0.13.1-cdh5.3.6.jar -rw-r--r-- 1 root supergroup 467908 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/sqoop/hive-hcatalog-core-0.13.1-cdh5.3.6.jar -rw-r--r-- 1 root supergroup 4695460 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/sqoop/hive-metastore-0.13.1-cdh5.3.6.jar -rw-r--r-- 1 root supergroup 767934 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/sqoop/hive-serde-0.13.1-cdh5.3.6.jar -rw-r--r-- 1 root supergroup 1908940 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/sqoop/hive-service-0.13.1-cdh5.3.6.jar -rw-r--r-- 1 root supergroup 7424 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/sqoop/hive-shims-0.13.1-cdh5.3.6.jar -rw-r--r-- 1 root supergroup 49468 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/sqoop/hive-shims-0.23-0.13.1-cdh5.3.6.jar -rw-r--r-- 1 root supergroup 43371 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/sqoop/hive-shims-common-0.13.1-cdh5.3.6.jar -rw-r--r-- 1 root supergroup 78719 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/sqoop/hive-shims-common-secure-0.13.1-cdh5.3.6.jar -rw-r--r-- 1 root supergroup 12990 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/sqoop/hive-shims-scheduler-0.13.1-cdh5.3.6.jar -rw-r--r-- 1 root supergroup 706710 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/sqoop/hsqldb-1.8.0.10.jar -rw-r--r-- 1 root supergroup 433368 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/sqoop/httpclient-4.2.5.jar -rw-r--r-- 1 root supergroup 227708 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/sqoop/httpcore-4.2.5.jar -rw-r--r-- 1 root supergroup 35058 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/sqoop/jackson-annotations-2.3.0.jar -rw-r--r-- 1 root supergroup 197986 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/sqoop/jackson-core-2.3.1.jar -rw-r--r-- 1 root supergroup 914311 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/sqoop/jackson-databind-2.3.1.jar -rw-r--r-- 1 root supergroup 201124 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/sqoop/jdo-api-3.0.1.jar -rw-r--r-- 1 root supergroup 125573 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/sqoop/jersey-servlet-1.14.jar -rw-r--r-- 1 root supergroup 1681148 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/sqoop/jetty-all-7.6.0.v20120127.jar -rw-r--r-- 1 root supergroup 87325 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/sqoop/jline-0.9.94.jar -rw-r--r-- 1 root supergroup 12131 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/sqoop/jpam-1.1.jar -rw-r--r-- 1 root supergroup 33015 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/sqoop/jsr305-1.3.9.jar -rw-r--r-- 1 root supergroup 15071 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/sqoop/jta-1.1.jar -rw-r--r-- 1 root supergroup 357991 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/sqoop/kite-data-core-0.15.0-cdh5.3.6.jar -rw-r--r-- 1 root supergroup 25242 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/sqoop/kite-hadoop-compatibility-0.15.0-cdh5.3.6.jar -rw-r--r-- 1 root supergroup 275186 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/sqoop/libfb303-0.9.0.jar -rw-r--r-- 1 root supergroup 348651 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/sqoop/libthrift-0.9.0-cdh5-2.jar -rw-r--r-- 1 root supergroup 388864 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/sqoop/mail-1.4.jar -rw-r--r-- 1 root supergroup 957162 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/sqoop/netty-3.4.0.Final.jar -rw-r--r-- 1 root supergroup 16989 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/sqoop/oozie-sharelib-sqoop-4.0.0-cdh5.3.6.jar -rw-r--r-- 1 root supergroup 19827 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/sqoop/opencsv-2.3.jar -rw-r--r-- 1 root supergroup 49342 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/sqoop/parquet-avro-1.5.0-cdh5.3.6.jar -rw-r--r-- 1 root supergroup 871534 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/sqoop/parquet-column-1.5.0-cdh5.3.6.jar -rw-r--r-- 1 root supergroup 20260 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/sqoop/parquet-common-1.5.0-cdh5.3.6.jar -rw-r--r-- 1 root supergroup 277472 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/sqoop/parquet-encoding-1.5.0-cdh5.3.6.jar -rw-r--r-- 1 root supergroup 352114 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/sqoop/parquet-format-2.1.0-cdh5.3.6.jar -rw-r--r-- 1 root supergroup 20382 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/sqoop/parquet-generator-1.5.0-cdh5.3.6.jar -rw-r--r-- 1 root supergroup 182816 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/sqoop/parquet-hadoop-1.5.0-cdh5.3.6.jar -rw-r--r-- 1 root supergroup 914345 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/sqoop/parquet-jackson-1.5.0-cdh5.3.6.jar -rw-r--r-- 1 root supergroup 995968 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/sqoop/snappy-java-1.0.4.1.jar -rw-r--r-- 1 root supergroup 1011420 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/sqoop/sqoop-1.4.5-cdh5.3.6.jar -rw-r--r-- 1 root supergroup 148627 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/sqoop/stringtemplate-3.2.1.jar -rw-r--r-- 1 root supergroup 94672 2018-09-06 21:27 /user/root/share/lib/lib_20180906212707/sqoop/xz-1.0.jar drwxr-xr-x - root supergroup 0 2018-09-06 23:28 /user/root/yinzhengjie-oozie-jobs drwxr-xr-x - root supergroup 0 2018-09-06 23:28 /user/root/yinzhengjie-oozie-jobs/shell -rw-r--r-- 1 root supergroup 277 2018-09-06 23:28 /user/root/yinzhengjie-oozie-jobs/shell/blog.sh -rw-r--r-- 1 root supergroup 1104 2018-09-06 23:28 /user/root/yinzhengjie-oozie-jobs/shell/job.properties -rw-r--r-- 1 root supergroup 1943 2018-09-06 23:28 /user/root/yinzhengjie-oozie-jobs/shell/workflow.xml [root@s101 oozie-4.0.0-cdh5.3.6]#
8>.执行任务（温馨提示：--oozie指定的是oozie的通讯地址，-config指定的是hdfs上的路径，相对路径前面回头一个默认的/user/root/）
[root@s101 oozie-4.0.0-cdh5.3.6]# bin/oozie job --oozie http://s101:11000/oozie -config yinzhengjie-oozie-jobs/shell/job.properties -run job: 0000000-180906233840835-oozie-root-W [root@s101 oozie-4.0.0-cdh5.3.6]#
9>.任务执行成功后查看是否有日志生成
[root@s101 ~]# cat /home/yinzhengjie/data/access.log 2018-09-06 22:23:36 : 尹正杰到此一游！ 2018-09-06 22:23:46 : 尹正杰到此一游！ [root@s101 ~]#
10>.杀死job任务
[root@s101 oozie-4.0.0-cdh5.3.6]# bin/oozie job --oozie http://s101:11000/oozie -kill 0000001-180906233840835-oozie-root-W [root@s101 oozie-4.0.0-cdh5.3.6]#
查看webUI状态：
11>.检查作业状态
[root@yinzhengjie oozie-4.0.0-cdh5.3.6]# bin/oozie job --oozie http://s101:11000/oozie -info 0000002-180907200401168-oozie-root-W Job ID : 0000002-180907200401168-oozie-root-W ------------------------------------------------------------------------------------------------------------------------------------ Workflow Name : shell-wf App Path : hdfs://s101:8020/user/root/yinzhengjie-oozie-jobs/shell Status : RUNNING Run : 0 User : root Group : - Created : 2018-09-08 00:36 GMT Started : 2018-09-08 00:36 GMT Last Modified : 2018-09-08 00:36 GMT Ended : - CoordAction ID: - Actions ------------------------------------------------------------------------------------------------------------------------------------ ID Status Ext ID Ext Status Err Code ------------------------------------------------------------------------------------------------------------------------------------ 0000002-180907200401168-oozie-root-W@shell-node PREP - - - ------------------------------------------------------------------------------------------------------------------------------------ 0000002-180907200401168-oozie-root-W@:start: OK - OK - ------------------------------------------------------------------------------------------------------------------------------------ [root@yinzhengjie oozie-4.0.0-cdh5.3.6]#
12>.查看日志
[root@yinzhengjie oozie-4.0.0-cdh5.3.6]# bin/oozie job --oozie http://s101:11000/oozie -log 0000002-180907200401168-oozie-root-W 2018-09-07 20:36:21,365 INFO ActionStartXCommand:541 - SERVER[] USER[root] GROUP[-] TOKEN[] APP[shell-wf] JOB[0000002-180907200401168-oozie-root-W] ACTION[0000002-180907200401168-oozie-root-W@:start:] Start action [0000002-180907200401168-oozie-root-W@:start:] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10] 2018-09-07 20:36:21,366 INFO ActionStartXCommand:541 - SERVER[] USER[root] GROUP[-] TOKEN[] APP[shell-wf] JOB[0000002-180907200401168-oozie-root-W] ACTION[0000002-180907200401168-oozie-root-W@:start:] [***0000002-180907200401168-oozie-root-W@:start:***]Action status=DONE 2018-09-07 20:36:21,366 INFO ActionStartXCommand:541 - SERVER[] USER[root] GROUP[-] TOKEN[] APP[shell-wf] JOB[0000002-180907200401168-oozie-root-W] ACTION[0000002-180907200401168-oozie-root-W@:start:] [***0000002-180907200401168-oozie-root-W@:start:***]Action updated in DB! 2018-09-07 20:36:21,435 INFO ActionStartXCommand:541 - SERVER[] USER[root] GROUP[-] TOKEN[] APP[shell-wf] JOB[0000002-180907200401168-oozie-root-W] ACTION[0000002-180907200401168-oozie-root-W@shell-node] Start action [0000002-180907200401168-oozie-root-W@shell-node] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10] [root@yinzhengjie oozie-4.0.0-cdh5.3.6]#
13>.检查xml文件是否合规
[root@yinzhengjie oozie-4.0.0-cdh5.3.6]# bin/oozie validate yinzhengjie-oozie-jobs/shell/workflow.xml Valid workflow-app [root@yinzhengjie oozie-4.0.0-cdh5.3.6]#
关于更多oozie的客户端命令请参考：https://yq.aliyun.com/articles/60212.
最后
以上就是眼睛大花生为你收集整理的Hadoop生态圈-Oozie实战之调度shell脚本的全部内容，希望文章能够帮你解决Hadoop生态圈-Oozie实战之调度shell脚本所遇到的程序开发问题。
如果觉得靠谱客网站的内容还不错，欢迎将靠谱客网站推荐给程序员好友。
本图文内容来源于网友提供，作为学习参考使用，或来自网络收集整理，版权属于原作者所有。
发表评论 取消回复