02）Hive安装与配置1.MySQL安装与配置2.Hive安装部署3.Hive元数据配置到MySQL4.启动Hive5.Hive常用交互命令6.Hive 常见属性配置

74 阅读 0 评论 49 点赞

我是靠谱客的博主满意白羊，最近开发中收集的这篇文章主要介绍02）Hive安装与配置1.MySQL安装与配置2.Hive安装部署3.Hive元数据配置到MySQL4.启动Hive5.Hive常用交互命令6.Hive 常见属性配置，觉得挺不错的，现在分享给大家，希望可以做个参考。

概述

这是学习hive的第二个阶段，搭环境，开搞吧！

1.MySQL安装与配置

为什么需要mysql？

原因在于Hive默认使用的元数据库为derby，开启Hive之后占用元数据库，且不与其他客户端共享数据，如果想多窗口操作就会报错，操作比较局限。所以我们需要讲Hive的元数据地址改为MySQL，可支持多窗口操作；

1）检验是否已安装

CentOS6之前默认安装mysql

CentOS7.5之后默认安装mariadb,它是mysql的开源版本；

rpm -qa | grep mysql

rpm -qa | grep mariadb

[fengxq@hadoop102 ~]$ rpm -qa | grep mariadb
mariadb-libs-5.5.56-2.el7.x86_64

2）删除已安装版本

sudo rpm -e --nodeps mariadb-libs-5.5.56-2.el7.x86_64

[fengxq@hadoop102 ~]$ sudo rpm -e --nodeps mariadb-libs-5.5.56-2.el7.x86_64

3) 安装mysql包

上传tar包到/opt/software

#创建mysql-rpm文件夹
mkdir mysql-rpm

#解压缩tar包
tar -xvf mysql-5.7.28-1.el7.x86_64.rpm-bundle.tar -C ./mysql-rpm

#解压缩完成后查看
/opt/software/mysql-rpm
[fengxq@hadoop102 mysql-rpm]$ ls
mysql-community-client-5.7.28-1.el7.x86_64.rpm
mysql-community-common-5.7.28-1.el7.x86_64.rpm
mysql-community-devel-5.7.28-1.el7.x86_64.rpm
mysql-community-embedded-5.7.28-1.el7.x86_64.rpm
mysql-community-embedded-compat-5.7.28-1.el7.x86_64.rpm
mysql-community-embedded-devel-5.7.28-1.el7.x86_64.rpm
mysql-community-libs-5.7.28-1.el7.x86_64.rpm
mysql-community-libs-compat-5.7.28-1.el7.x86_64.rpm
mysql-community-server-5.7.28-1.el7.x86_64.rpm
mysql-community-test-5.7.28-1.el7.x86_64.rpm

注意：mysql需安装4个rpm包且有顺序要求。

mysql-community-common-5.7.28-1.el7.x86_64.rpm

mysql-community-libs-5.7.28-1.el7.x86_64.rpm

mysql-community-libs-compat-5.7.28-1.el7.x86_64.rpm

mysql-community-client-5.7.28-1.el7.x86_64.rpm

mysql-community-server-5.7.28-1.el7.x86_64.rpm

#按照安装顺序进行重命名
mv mysql-community-common-5.7.28-1.el7.x86_64.rpm 01_mysql-community-common-5.7.28-1.el7.x86_64.rpm

mv mysql-community-libs-5.7.28-1.el7.x86_64.rpm 02_mysql-community-libs-5.7.28-1.el7.x86_64.rpm

mv mysql-community-libs-compat-5.7.28-1.el7.x86_64.rpm 03_mysql-community-libs-compat-5.7.28-1.el7.x86_64.rpm

mv mysql-community-client-5.7.28-1.el7.x86_64.rpm 04_mysql-community-client-5.7.28-1.el7.x86_64.rpm

mv mysql-community-server-5.7.28-1.el7.x86_64.rpm 05_mysql-community-server-5.7.28-1.el7.x86_64.rpm

#查看
[fengxq@hadoop102 mysql-rpm]$ ls
01_mysql-community-common-5.7.28-1.el7.x86_64.rpm
02_mysql-community-libs-5.7.28-1.el7.x86_64.rpm
03_mysql-community-libs-compat-5.7.28-1.el7.x86_64.rpm
04_mysql-community-client-5.7.28-1.el7.x86_64.rpm
05_mysql-community-server-5.7.28-1.el7.x86_64.rpm
mysql-community-devel-5.7.28-1.el7.x86_64.rpm
mysql-community-embedded-5.7.28-1.el7.x86_64.rpm
mysql-community-embedded-compat-5.7.28-1.el7.x86_64.rpm
mysql-community-embedded-devel-5.7.28-1.el7.x86_64.rpm
mysql-community-test-5.7.28-1.el7.x86_64.rpm

依次安装rpm包

安装rpm包1

sudo rpm -ivh 01_mysql-community-common-5.7.28-1.el7.x86_64.rpm

[fengxq@hadoop102 mysql-rpm]$ sudo rpm -ivh 01_mysql-community-common-5.7.28-1.el7.x86_64.rpm
警告：01_mysql-community-common-5.7.28-1.el7.x86_64.rpm: 头V3 DSA/SHA1 Signature, 密钥 ID 5072e1f5: NOKEY
准备中...                          ################################# [100%]
正在升级/安装...
   1:mysql-community-common-5.7.28-1.e################################# [100%]

安装rpm包2

sudo rpm -ivh 02_mysql-community-libs-5.7.28-1.el7.x86_64.rpm

[fengxq@hadoop102 mysql-rpm]$ sudo rpm -ivh 02_mysql-community-libs-5.7.28-1.el7.x86_64.rpm
警告：02_mysql-community-libs-5.7.28-1.el7.x86_64.rpm: 头V3 DSA/SHA1 Signature, 密钥 ID 5072e1f5: NOKEY
准备中...                          ################################# [100%]
正在升级/安装...
   1:mysql-community-libs-5.7.28-1.el7################################# [100%]

安装rpm包3

sudo rpm -ivh 03_mysql-community-libs-compat-5.7.28-1.el7.x86_64.rpm

[fengxq@hadoop102 mysql-rpm]$ sudo rpm -ivh 03_mysql-community-libs-compat-5.7.28-1.el7.x86_64.rpm
警告：03_mysql-community-libs-compat-5.7.28-1.el7.x86_64.rpm: 头V3 DSA/SHA1 Signature, 密钥 ID 5072e1f5: NOKEY
准备中...                          ################################# [100%]
正在升级/安装...
   1:mysql-community-libs-compat-5.7.2################################# [100%]

安装rpm包4

sudo rpm -ivh 04_mysql-community-client-5.7.28-1.el7.x86_64.rpm

[fengxq@hadoop102 mysql-rpm]$ sudo rpm -ivh 04_mysql-community-client-5.7.28-1.el7.x86_64.rpm
警告：04_mysql-community-client-5.7.28-1.el7.x86_64.rpm: 头V3 DSA/SHA1 Signature, 密钥 ID 5072e1f5: NOKEY
准备中...                          ################################# [100%]
正在升级/安装...
   1:mysql-community-client-5.7.28-1.e################################# [100%]

安装rpm包5

sudo rpm -ivh 05_mysql-community-server-5.7.28-1.el7.x86_64.rpm

[fengxq@hadoop102 mysql-rpm]$ sudo rpm -ivh 05_mysql-community-server-5.7.28-1.el7.x86_64.rpm
警告：05_mysql-community-server-5.7.28-1.el7.x86_64.rpm: 头V3 DSA/SHA1 Signature, 密钥 ID 5072e1f5: NOKEY
错误：依赖检测失败：
	libaio.so.1()(64bit) 被 mysql-community-server-5.7.28-1.el7.x86_64 需要
	libaio.so.1(LIBAIO_0.1)(64bit) 被 mysql-community-server-5.7.28-1.el7.x86_64 需要
	libaio.so.1(LIBAIO_0.4)(64bit) 被 mysql-community-server-5.7.28-1.el7.x86_64 需要

通过yum安全缺少的依赖，然后重新安装此包

[fengxq@hadoop102 mysql-rpm]$ sudo yum install -y libaio

[fengxq@hadoop102 mysql-rpm]$ yum install -y libaio
已加载插件：fastestmirror
您需要 root 权限执行此命令。
[fengxq@hadoop102 mysql-rpm]$ sudo yum install -y libaio
已加载插件：fastestmirror
Determining fastest mirrors
epel/x86_64/metalink                                                           | 4.6 kB  00:00:00
 * base: mirrors.bfsu.edu.cn
 * epel: hkg.mirror.rackspace.com
 * extras: mirrors.bfsu.edu.cn
 * updates: mirrors.bfsu.edu.cn
base                                                                           | 3.6 kB  00:00:00
epel                                                                           | 4.7 kB  00:00:00
extras                                                                         | 2.9 kB  00:00:00
updates                                                                        | 2.9 kB  00:00:00
(1/2): epel/x86_64/updateinfo                                                  | 1.0 MB  00:00:02
(2/2): epel/x86_64/primary_db                                                  | 6.9 MB  00:00:22
正在解决依赖关系
--> 正在检查事务
---> 软件包 libaio.x86_64.0.0.3.109-13.el7 将被 安装
--> 解决依赖关系完成

依赖关系解决

======================================================================================================
 Package               架构                  版本                           源                   大小
======================================================================================================
正在安装:
 libaio                x86_64                0.3.109-13.el7                 base                 24 k

事务概要
======================================================================================================
安装  1 软件包

总下载量：24 k
安装大小：38 k
Downloading packages:
libaio-0.3.109-13.el7.x86_64.rpm                                               |  24 kB  00:00:00
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
警告：RPM 数据库已被非 yum 程序修改。
  正在安装    : libaio-0.3.109-13.el7.x86_64                                                      1/1
  验证中      : libaio-0.3.109-13.el7.x86_64                                                      1/1

已安装:
  libaio.x86_64 0:0.3.109-13.el7

完毕！

sudo rpm -ivh 05_mysql-community-server-5.7.28-1.el7.x86_64.rpm

[fengxq@hadoop102 mysql-rpm]$ sudo rpm -ivh 05_mysql-community-server-5.7.28-1.el7.x86_64.rpm
警告：05_mysql-community-server-5.7.28-1.el7.x86_64.rpm: 头V3 DSA/SHA1 Signature, 密钥 ID 5072e1f5: NOKEY
准备中...                          ################################# [100%]
正在升级/安装...
   1:mysql-community-server-5.7.28-1.e################################# [100%]

4）初始化数据库

sudo mysqld --initialize --user=mysql

5）查看临时生成的root用户密码

sudo cat /var/log/mysqld.log

[fengxq@hadoop102 etc]$ sudo cat /var/log/mysqld.log
2021-06-05T08:51:06.141377Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
2021-06-05T08:51:06.307927Z 0 [Warning] InnoDB: New log files created, LSN=45790
2021-06-05T08:51:06.334873Z 0 [Warning] InnoDB: Creating foreign key constraint system tables.
2021-06-05T08:51:06.398371Z 0 [Warning] No existing UUID has been found, so we assume that this is the first time that this server has been started. Generating a new UUID: 2a5a00d8-c5db-11eb-ad7e-000c2902dd36.
2021-06-05T08:51:06.412842Z 0 [Warning] Gtid table is not ready to be used. Table 'mysql.gtid_executed' cannot be opened.
2021-06-05T08:51:07.104485Z 0 [Warning] CA certificate ca.pem is self signed.
2021-06-05T08:51:07.516528Z 1 [Note] A temporary password is generated for root@localhost: deo9=iU&f(-5

6）启动mysql服务

sudo systemctl start mysqld

7) 登陆mysql数据库

mysql -uroot -p

[fengxq@hadoop102 mysql-rpm]$ mysql -uroot -p
Enter password:
Welcome to the MySQL monitor.  Commands end with ; or g.
Your MySQL connection id is 2
Server version: 5.7.28

Copyright (c) 2000, 2019, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or 'h' for help. Type 'c' to clear the current input statement.

mysql>

8) 必须重置root密码,否则会报错

mysql> show databases;
ERROR 1820 (HY000): You must reset your password using ALTER USER statement before executing this statement.

password()为加密函数

set password = password("123456");

9）远程连接设置

一般安装完成后，需要进行2个设置：

1）字符集，因为我们mysql只用来存储hive的元数据，都是英文，所以用默认字符集即可；

2）远程连接需要进行设置，默认不支持

客户端连接尝试

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-BiNYpEVx-1622943182627)(D:2_学习15-大数据2_大数据基础阶段2_尚硅谷大数据技术之Hive5.总结hive安装与配置.assets1622885097755.png)]

查看远程连接

mysql> use mysql;

查看root的远程连接为本地连接

mysql> select host, user, authentication_string from user;
+-----------+---------------+-------------------------------------------+
| host      | user          | authentication_string                     |
+-----------+---------------+-------------------------------------------+
| localhost | root          | *6BB4837EB74329105EE4568DDA7DC67ED2CA2AD9 |
| localhost | mysql.session | *THISISNOTAVALIDPASSWORDTHATCANBEUSEDHERE |
| localhost | mysql.sys     | *THISISNOTAVALIDPASSWORDTHATCANBEUSEDHERE |
+-----------+---------------+-------------------------------------------+
3 rows in set (0.00 sec)

设置成本地和远程都可以链接方式：%

update user set host= '%' where user = 'root';

mysql> update user set host= '%' where user = 'root';
Query OK, 1 row affected (0.00 sec)
Rows matched: 1  Changed: 1  Warnings: 0

查看已改过来了

mysql> select host, user, authentication_string from user;
+-----------+---------------+-------------------------------------------+
| host      | user          | authentication_string                     |
+-----------+---------------+-------------------------------------------+
| %         | root          | *6BB4837EB74329105EE4568DDA7DC67ED2CA2AD9 |
| localhost | mysql.session | *THISISNOTAVALIDPASSWORDTHATCANBEUSEDHERE |
| localhost | mysql.sys     | *THISISNOTAVALIDPASSWORDTHATCANBEUSEDHERE |
+-----------+---------------+-------------------------------------------+
3 rows in set (0.01 sec)

刷新权限

flush privileges;

mysql> flush privileges;
Query OK, 0 rows affected (0.04 sec)

客户端再次连接尝试

mysql安装成功

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-nGZTTk5O-1622943182628)(D:2_学习15-大数据2_大数据基础阶段2_尚硅谷大数据技术之Hive5.总结hive安装与配置.assets1622886121553.png)]

2.Hive安装部署

1）上传解压

tar -zxvf apache-hive-3.1.2-bin.tar.gz -C ../module/

2）重命名

mv apache-hive-3.1.2-bin/ hive-3.1.2

3）配置环境变量

#声明HIVE_HOME变量
HIVE_HOME/opt/module/hive-3.1.2

# 打开配置环境变量
sudo vim /etc/profile.d/my_env.sh

添加HIVE_HOME环境变量

#声明一个JAVA_HOME变量
JAVA_HOME=/opt/module/jdk1.8.0_212

#声明HADOOP_HOME变量
HADOOP_HOME=/opt/module/ha/hadoop-3.1.3

#声明HIVE_HOME变量
HIVE_HOME=/opt/module/hive-3.1.2

#声明PATH变量
PATH=$PATH:$JAVA_HOME/bin:$HADOOP_HOME/bin:$HADOOP_HOME/sbin:$HIVE_HOME/bin

#提升全局变量
export JAVA_HOME HADOOP_HOME HIVE_HOME PATH

4）source使其生效

source /etc/profile

5）解决日志Jar包冲突

删除log4j-slf4j-impl-2.10.0.jar包

mv $HIVE_HOME/lib/log4j-slf4j-impl-2.10.0.jar $HIVE_HOME/lib/log4j-slf4j-impl-2.10.0.jar.bak

3.Hive元数据配置到MySQL

1）拷贝驱动

将mysql的jdbc驱动拷贝到hive的lib目录下

cp /opt/software/mysql-connector-java-5.1.48.jar $HIVE_HOME/lib

2）配置MetaStore到mysql

在$HIVE_HOME/conf目录下新建hive-site.xml文件

[fengxq@hadoop102 conf]$ pwd
/opt/module/hive-3.1.2/conf
[fengxq@hadoop102 conf]$ vim hive-site.xml

添加如下内容

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
    <!-- jdbc连接的URL -->
    <property>
        <name>javax.jdo.option.ConnectionURL</name>
        <value>jdbc:mysql://hadoop102:3306/metastore?useSSL=false</value>
</property>

    <!-- jdbc连接的Driver-->
    <property>
        <name>javax.jdo.option.ConnectionDriverName</name>
        <value>com.mysql.jdbc.Driver</value>
</property>

	<!-- jdbc连接的username-->
    <property>
        <name>javax.jdo.option.ConnectionUserName</name>
        <value>root</value>
    </property>

    <!-- jdbc连接的password -->
    <property>
        <name>javax.jdo.option.ConnectionPassword</name>
        <value>123456</value>
</property>

    <!-- Hive默认在HDFS的工作目录 -->
    <property>
        <name>hive.metastore.warehouse.dir</name>
        <value>/user/hive/warehouse</value>
    </property>
    
   <!-- Hive元数据存储的验证 -->
    <property>
        <name>hive.metastore.schema.verification</name>
        <value>false</value>
    </property>
   
    <!-- 元数据存储授权  -->
    <property>
        <name>hive.metastore.event.db.notification.api.auth</name>
        <value>false</value>
    </property>
</configuration>

4.启动Hive

1）初始化元数据库

-新建hive元数据库

mysql> create database metastore;
Query OK, 1 row affected (0.00 sec)

-初始化Hive元数据库

schematool -initSchema -dbType mysql -verbose

2)启动Hive

-先启动hadoop集群

#这里通过脚本启动，包括启动hdfs集群和yarn集群
[fengxq@hadoop102 etc]$ my_cluster.sh start

-启动hive

hive

[fengxq@hadoop102 hive-3.1.2]$ bin/hive
which: no hbase in (/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/module/jdk1.8.0_212/bin:/opt/module/hadoop-3.1.3/bin:/opt/module/hadoop-3.1.3/sbin:/opt/module/hive-3.1.2/bin:/home/fengxq/.local/bin:/home/fengxq/bin)
Hive Session ID = d792a300-a7cf-4167-bf3a-f8e78714e2f5

Logging initialized using configuration in jar:file:/opt/module/hive-3.1.2/lib/hive-common-3.1.2.jar!/hive-log4j2.properties Async: true
Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. spark, tez) or using Hive 1.X releases.
Hive Session ID = 62165c44-ccf8-4be7-96f8-a304ee3dcbf9

-使用hive

hive> show databases;
OK
default
Time taken: 0.589 seconds, Fetched: 1 row(s)

hive> show tables;
OK
Time taken: 0.05 seconds

hive> create table test(id int, name string);
OK
Time taken: 0.525 seconds

hive> show tables;
OK
test

hive> insert into test values(1001, fengxq);
Time taken: 29.479 seconds # hive创建表是通过mapreduce创建的，所花时间较长，不推荐使用此方法创建数据

3）是用元数据服务方式访问hive

-在hive-sit.xml中添加配置信息

<!-- 指定存储元数据要连接的地址 -->
<property>
	<name>hive.metastore.uris</name>
	<value>thrift://hadoop102:9083</value>
</property>

-启动metastore

注意：启动后窗口不能关闭和再操作，需要打开新的shell窗口做别的操作

hive --service metastore

[fengxq@hadoop102 hive-3.1.2]$ hive --service metastore
2021-06-06 06:47:53: Starting Hive Metastore Server

–启动hive

[fengxq@hadoop102 hive-3.1.2]$ ./bin/hive

4）使用JDBC的方式访问Hive

-在hive-site.xml文件中添加如下配置信息

<!-- 指定hiveserver2连接的host -->
<property>
	<name>hive.server2.thrift.bind.host</name>
	<value>hadoop102</value>
</property>

<!-- 指定hiveserver2连接的端口号 -->
<property>
	<name>hive.server2.thrift.port</name>
	<value>10000</value>
</property>

-启动hiveserver2

此窗口不能关闭也不能进行其他操作，想要其他操作只能另起窗口；

.bin/hive --service hiveserver2

[fengxq@hadoop102 hive-3.1.2]$ ./bin/hive --service hiveserver2
which: no hbase in (/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/module/jdk1.8.0_212/bin:/opt/module/hadoop-3.1.3/bin:/opt/module/hadoop-3.1.3/sbin:/opt/module/hive-3.1.2/bin:/home/fengxq/.local/bin:/home/fengxq/bin)
2021-06-06 06:58:26: Starting HiveServer2
Hive Session ID = 26f4e39b-6d68-48a9-a0f9-979ba2a09ab4
Hive Session ID = e7e93cbe-ee09-4b88-b956-b0387de75688

–启动beeline客户端（需要多等一会儿）

./bin/beeline -u jdbc:hive2://hadoop102:10000 -n fengxq

beeline是通过hiveserver2服务连接hive

查询界面更友好写

0: jdbc:hive2://hadoop102:10000> show tables;
INFO  : Concurrency mode is disabled, not creating a lock manager
+-----------+
| tab_name  |
+-----------+
| test      |
+-----------+

0: jdbc:hive2://hadoop102:10000> select * from test;
INFO  : Concurrency mode is disabled, not creating a lock manager
+----------+------------+
| test.id  | test.name  |
+----------+------------+
| 1001     | fengxq     |
+----------+------------+

退出beeline客户端

！quit 没有分号

0: jdbc:hive2://hadoop102:10000> !quit

4）解决metastore和hiveserver2阻塞问题

-shell命令介绍

前台启动的方式导致需要打开多个shell窗口，如何解决？

一般会组合使用: nohup [xxx命令操作] 1> file 2>&1 & ，表示将xxx命令运行的

nohup:放在命令开头，表示不挂起，也就是即使关闭终端进程也继续保持进行运行状态

0：标准输入

1：标准输出

2：错误输出

&：放在命令结尾，表示后台运行

2>&1：表示将错误重定向到标准输出上

# 创在两个日志文件，用存储这两个服务的日志
mkdir ./logs
touch ./logs/metastore.log
touch ./logs/hivesterver2.log

–起服务命令

nohup hive --service hiveserver2 1>/opt/module/hive-3.1.2/logs/hivesterver2.log 2>&1 &

nohup hive --service metastore 1>/opt/module/hive-3.1.2/logs/metastore.log 2>&1 &

–停服务命令

sudo netstat -anp |grep 9083 |awk '{print $7}' | awk -F / '{print $1}' |xargs -n1 kill -9

sudo netstat -anp |grep 10000 |awk '{print $7}' | awk -F / '{print $1}' |xargs -n1 kill -9

-编写脚本

vim $HIVE_HOME/bin/hiveservices.sh

#!/bin/bash
HIVE_LOG_DIR=$HIVE_HOME/logs
if [ ! -d $HIVE_LOG_DIR ]
then
	mkdir -p $HIVE_LOG_DIR
fi
#检查进程是否运行正常，参数1为进程名，参数2为进程端口
function check_process()
{
    pid=$(ps -ef 2>/dev/null | grep -v grep | grep -i $1 | awk '{print $2}')
    ppid=$(netstat -nltp 2>/dev/null | grep $2 | awk '{print $7}' | cut -d '/' -f 1)
    echo $pid
    [[ "$pid" =~ "$ppid" ]] && [ "$ppid" ] && return 0 || return 1
}

function hive_start()
{
    metapid=$(check_process HiveMetastore 9083)
    cmd="nohup hive --service metastore >$HIVE_LOG_DIR/metastore.log 2>&1 &"
    cmd=$cmd" sleep 4; hdfs dfsadmin -safemode wait >/dev/null 2>&1"
    [ -z "$metapid" ] && eval $cmd || echo "Metastroe服务已启动"
    server2pid=$(check_process HiveServer2 10000)
    cmd="nohup hive --service hiveserver2 >$HIVE_LOG_DIR/hiveServer2.log 2>&1 &"
    [ -z "$server2pid" ] && eval $cmd || echo "HiveServer2服务已启动"
}

function hive_stop()
{
    metapid=$(check_process HiveMetastore 9083)
    [ "$metapid" ] && kill $metapid || echo "Metastore服务未启动"
    server2pid=$(check_process HiveServer2 10000)
    [ "$server2pid" ] && kill $server2pid || echo "HiveServer2服务未启动"
}

case $1 in
"start")
    hive_start
    ;;
"stop")
    hive_stop
    ;;
"restart")
    hive_stop
    sleep 2
    hive_start
    ;;
"status")
    check_process HiveMetastore 9083 >/dev/null && echo "Metastore服务运行正常" || echo "Metastore服务运行异常"
    check_process HiveServer2 10000 >/dev/null && echo "HiveServer2服务运行正常" || echo "HiveServer2服务运行异常"
    ;;
*)
    echo Invalid Args!
    echo 'Usage: '$(basename $0)' start|stop|restart|status'
    ;;
esac

-添加执行权限

chmod u+x $HIVE_HOME/bin/hiveservices.sh

-启动后台服务

[fengxq@hadoop102 hive-3.1.2]$ hiveservices.sh start

5.Hive常用交互命令

1）“-e” 不进入hive的交互窗口执行sql语句

[fengxq@hadoop102 hive-3.1.2]$ ./bin/hive 'select * from test';

2）“-f” 执行脚本中的sql语句

vim ./datas/hivef.sql
#写入
select * from test;

#执行文件中的sql
./bin/hive -f ./datas/hivef.sql

#执行文件中的sql并写结果文件
./bin/hive -f ./datas/hivef.sql ./datas/hivef_result.txt

3)hive 命令窗口查看hdfs文件

登陆hive命令窗口

hive> dfs -ls /;
Found 2 items
drwxrwx---   - fengxq supergroup          0 2021-06-06 06:19 /tmp
drwxr-xr-x   - fengxq supergroup          0 2021-06-06 06:22 /user

4)hive 命令窗口查看linux本地文件

hive> !ls /;
bin
boot
dev
etc
home
lib
lib64
lost+found
media
mnt
opt
proc
root
run
sbin
srv
sys
tmp
usr
var

6.Hive 常见属性配置

1）打印当前库和表头

在hive-site.xml添加如下两个配置

<!--添加打印表头-->
<property>
	<name>hive.cli.print.header</name>
	<value>true</value>
</property>
<!--添加显示当前库-->
<property>
	<name>hive.cli.print.current.db</name>
	<value>true</value>
</property>

hive (default)> select * from test;
OK
test.id	test.name
1001	fengxq
Time taken: 1.883 seconds, Fetched: 1 row(s)

2）Hive运行日志信息配置

-hive的默认日志

根据hive-log4j2.properties.template文件找到

property.hive.log.dir = ${sys:java.io.tmpdir}/$ {sys:user.name}

查看默认的hive日志放在/tmp/fengxq/hive.log(当前用户名)

-修改hive的log存放目录

配置文件去掉.template后缀

[fengxq@hadoop102 conf]$ mv hive-log4j2.properties.template hive-log4j2.properties

修改hive的log存放到 $HIVE_HOME/logs/下

vim hive-log4j2.properties

property.hive.log.dir = $HIVE_HOME/logs

-参数配置方式

查看当前所有的配置信息

hive (default)> set;

– 参数的3中配置方式

（1）配置文件方式

默认配置文件方式：hive-default.xml

用户自定义配置文件：hive-site.xml

另外，Hive会读Hadoop的配置文件，因为Hive是作为Hadoop的客户端启动的，所以Hive的配置文件会覆盖Hadoop的配置。

注意：配置文件是对本机启动的所有Hive进程都有效的

(2) 命令行参数方式

-hiveconf param=value来设定参数(等号两边不能有空格)

注意：此次配置仅对本次hive启动有效

比如：

./bin/hive -hiveconf mapred.reduce.tasks=10;

查看参数

set mapred.reduce.tasks;

（3）参数声明方式

可以在HQL中使用set关键字设定参数

注意：也是仅对本次hive启动有效

比如：

hive (default)> set mapred.reduce.tasks=20;

查看：

hive (default)> set mapred.reduce.tasks;
mapred.reduce.tasks=20

Hive的环境搭建到此已经搭建完成，如果对您有用，请帮忙点个赞，让我知道你在看！！！

最后

以上就是满意白羊为你收集整理的02）Hive安装与配置1.MySQL安装与配置2.Hive安装部署3.Hive元数据配置到MySQL4.启动Hive5.Hive常用交互命令6.Hive 常见属性配置的全部内容，希望文章能够帮你解决02）Hive安装与配置1.MySQL安装与配置2.Hive安装部署3.Hive元数据配置到MySQL4.启动Hive5.Hive常用交互命令6.Hive 常见属性配置所遇到的程序开发问题。

如果觉得靠谱客网站的内容还不错，欢迎将靠谱客网站推荐给程序员好友。

本图文内容来源于网友提供，作为学习参考使用，或来自网络收集整理，版权属于原作者所有。

本文分类：Hadoop
浏览次数：74 次浏览
发布日期：2023-10-08 12:55:17
本文链接：https://www.kaopuke.com/article/k-p-k_14_uzokfy_13_j_10_y.html